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This' report' concludes a two-year study of human services informatiofK 
clearinghouses conducted by Applied Managementf' Sciences^. Ihc^ of Silver Spring, 
Maryland and its subcontractor, ^Cuadra Associates, Inc. of ^fianta Monica, 
ealifornia. The study was conducted ior th6 Qffice.of the Assistant Secretary for 
Planning and Evaluation of the Department of Health and Human Servicec. 
^ ' \ ' ' / - • • 

Histdry-of the Study^ 

^ ' •» * ^ • ■ 

The study was intended to provide an information base about human services 
information clearinghouses that would assist decision^makers^and program managers 
in planning, operating^ ajid?;evaluating these-services. It was performed in three 
phases, each. responding to different needs. Phase I produced a working definition of 
the term "information clearinghouse^.' that was used to identify existing human ^ 
services information clearinghouses and related organizations. Phase II- provided 
descriptive information about management, structCire, operating procedures^ costs, 
and^ evaluation for.' a sample of 27 information clearinghouses and 'five related^ 
irjformation service organizations. It provided the first empiripar information about 
the operations of a* large number of government-supported information 
clearinghouses. THe results of the first two phases of the study are presented in 
three major publications: " • ' . " ^ 

g Conceptual Framework/ Definition - of' Clearinghouses, aqd . 
Typology of Clearinghouses: A Working Paper, an unpgblished 
paper dated March i58U; "T"^'^ ^"V-— ] 

m Catalog, of . .Human Services Information Re30urc?e 
Organizations, June 1980,, and available frjoni Project. Share as 
Monograph No. 15; and ' , • 



■I 



/J 



Descriptive Analysis ■ of Human Services ^ Information 
Clearinghouses, January 1981, available from NTIS (PB 
81rl69997;. ' - \ " . 
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t|ii8 research indicat^ed thaf there was no generally accepted definition of the 
t^rm ; *Mnformation clearinghouse.*^ Concepts of: \^hat a clearinghouse is and 
actimies it should perform have evolved over the past twenty years as needs have 
changed and technological advanc^es have n;)ade new'services possible. The concept 
of a "clearinghouse^'^varied also by thfe needs 'of the^program office sponsoring the 
j^vice anostiy th e othe r informa tion resour ces, available in t]iat_lLeli.^_^^ 
purpose of tn|s ^.^tudy, seven criteria 'were established that specified the minimuni 
characteristics N^n organization had to meet to be classified « as a clearinghouse. 
They were: 

Must ^defin^its focus in ternhs-of .a specific subject area and * ^ 
* target'audienc;e* ^ ' ' . ^ 

Must aptively^vl^gage, in the acquisition of literature-based* 
information related^^^^to its fo<^s or maintain a database v 
^ representing je^qrro oNiteraturMsased^information resdbrces. 

\ Must process and^ganizeHhe acquired information into a 
collection to provid^oi^ystamatic^eTarcfi and access. 

Must be willing and able to accept ^ihdlyiduat inquiries with \ 
minimal requirements related to the form df-^the inquiry. 

r / , ■ ' ' ' . • k ^ ^\ ■ 

Must be^willing and able to respond to each inquiry^ in a form 

tailored bd the request. • v 

f * ^ , ~ " 

Must be willing ahd-able to conduct systematic searches of its. 
information collection in response to inquiries. 

-Must haye \an outreach component ^^^r communicating- with 
users and a dissemination component to inform the audience of. 
information available in or frdnn the clearinghouse. 

A list oT>p^ential:;human\ser vices information (Clearinghouses was generated 
from a. review of^ ciire^ories^^aqd guides to information services. ^Each potential 
Qlearingliouse was cipntacted by mail^to obtain publicly available information* about 
ijt% programs aj:id services. On the basis of this information, organizations were 
classified as information clearinghouses or related types of information service 
organizations. \ 

• J-. 
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The sUi\yey^ fpuhcl that/ as of June 1980^ there Were 71 human services 
information clearinghouses and 84 other information resource organizations that 
were similar to, but did not meet all the criteria to be classified as, information' , 

"^clearinghouses. Ninety-Height of the organizations identified were funded by Fedei^al 
agencies; 43 were funded by private organizations, .and the remainder were, funded 
by academip institutions, state governments, or .^Ccal governments. Of the 50 
Federally^ funded organizations that reported budget information, 3^, or 66 percent, 

-were'operating*orra1^udget-of-$500jOOO^per7ear or^less.— 

In Phase II, site visits to 27 human services information clearinghouses dnd 5/^ 
related information resource organizations were conducted to obtain more 
Jriformatioli about operating procedures. .This priase revealed that a clearinghouse is* . 
a, cbmplex entity performing many ^ifferent and interrelated tasks. How each 
clearinghouse performed these activities depended, in part, on contextual factors ^ 
such as the misjion of the clearinghouse (e.g., whether it was intended to serve as a 
^neutral .information resource, or as an advgc^te for a particular 'position), the type 
of audiencJe it Iwas designed to reach" (e.g., researchers, practitioners, or the lay 
public), Vncl the relationship of the clearinghouse to» a parent or sponsoring agen^ 
(e.g., autonohious clearinghouse versus one that was an integrated component 
within/a dissemination or programmatic unit of .a Federal agency). .Other factors^* 
influencing the design and operation of the clearinghouse included the. year t/ie 
clearinghouse was organized and the history of its mission (i.e.,^lder^learinghouses . 
werej founded on somewhat different prenniseSothan newer clearinghouses) and its 
^ subject focus (e.g., there appeared to be ln|ierent differences be^^en the^/health, 
education, and social services fields that ^affected the structure/and focus^ of 

information /disseminatldn activities). These factors formed an ehyironment that 

0 / " - • * J / ' ^ 

partially determined the structure an.d design- of each clearinghouse. 

■ ■ V ' ' ' ' " • ' ' '■■ / 

Phsse III purpQge and Methodology ' " . ' ' ' . ■ * 

Phydse in was/ initiated in Dei^bmber 1980, to address selected policy-issues 
pertaining to the planning, operation, and evaluation of humafn services information 
blear/nghpuses. Its intent is to provide policyro^kers with arj understanding of the 
background'of major policy topics being addressed by the government, an awareness 
of ^^le implications of proposed. policy options, and suggestions for potential courses 
of action. . 



The policy* topics addressed in Phase III /were, selected by a Department bf> 
- Health and Human Services advisory committee for this study^ Topics were*selecled 
because of their relevance to current discussions within the^Oepartment and their 
' importance to the planning of new^clearinghouses and trie evaluation of existing 
\on^s. the topics selected were: factors to cQnsider in panning a clearinghouse cost 
recovery in clearinghouse * opera tibp^ overlap and duplication among information 
. • service^^the rol^^ of clearinghouse publications wogranris; and techniques and 
measui^ ir^for^evaluating ~clea ringhoMses> :§^!fi^®}Y^^^^ A*^.®^? 

topics, an introductory chapter was added on the role clearinghouses can play in 
^ information dissemination and "diffusion. ' . 

/ The information presented in this. report is derived from' a variety. of*sources« A 
literature xeview was conducted to learn about previous research* on the policy 
' topics. Interviews were conducted with selected private information services* and 
commercial ^Wishers to understand how they operate and what th^ir concerns are. 
./* Several clearinghouse directors were also contacted for* information or to clarify 
opinions they had shared during our Phase II interviews. The data collected 
II were* reviewed and, where appropriate, re-anaiyzed to support this phase 



of the 



e work. In addition, two. group meetings were held to help clarify the issues and 
to /obtain feedback on' pur ' perceptions of the problems; the first meeting was 
. ^ attended by conjtractors who .operate government-supported clearinghouses and the 
^ ^ second was attended by Federal officials who monitor qf operate clearinghouses. 

\^^^ - ./'•>■ 
. ^ The cesults.of this research are presented in this report. We have attempted to 

make thfe document useful to policyrnakers by exploring topics of current concern to 

the Department and the Administration. Where appropriaterwe have gone beyond 

presenting findings .by stating our conclusions in a clearly . identified summary 

chapter at the end of the report. The conclusions are those of the study team and 

J are offered to DHHS for its consideration. We are 'hopeful that the ensuing 

document will meet the needs of government officials involved in managing human 

services information clearinghouses. \ ^ ' ' 
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THE ROLE OF CLEARINGHOUSES IN INFORMATION 

DISSEMINATION AND DIFFUSION ■ / 

Introduction , ^ 

^hc^ intent of this chapter is to provide a framework for/establishing realistic 
expectations for information clearinghouses in the human services area and to place 
policies for their continued operatipn within that cbntaxt. It begins with an 
overview of the variety that exists amlong the organizations now known as 
"information clearinghouses^" The chapter also establishes the background for the 
report by describing the role' clearinghouses can play in the knowledge utilization 
process* 

Variations in he Concept of a Clearinghouse 

One of the major findings from our earlier analysis of 27 clearinghouses is that 
there is no cle^r, precise, .or unitary concept of what a clearinghouse is. The tern 
can beft and has been, used to refer to a wide variety of information «^vice 
organizations. The variety i.% in part, a function of the linkages that clearinghouse 
personnel establish with an agency program area and with other inforiT^tion services 
in the field. Jhis section describes some of this variation to help-readers develop ah 
Understanding, of %.dif ferences that must be' considered when policies for\ 
clearinghouses are being discussed. 

Exhibit Ul presents a ^conceptual diagram 0/ the complete information 
utilization process. A variety of organizations, including libraries, publishers, 
anforfiiation centers, and information linkage agents, can and do participate in the- 
process. Clearinghouses perform a unique role in the process because they can take 
or! some or all of the functions of several different types of information services. 
Clearinghouses can resemble: 

• libraries and information centers (in their provision of reference and 
/ referral services, including literature searches for requesters);' 

1.1 • 
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EXUIOIT 1.1 CONCEPTUAL DIAGRAM (T WK IHFORHAl ION /TRANSFER AND UTILI2/ 
PROCESS 1/ • / I 




Adapled frfniK Lancaster; 1977:3. 



- • special' iibraries' (in'- their collection of materials in a 
"natrowly * defined area - .andu Jn_ J:he_ _j3[rep^ of 
biyiogrkphies within that scope); 

• seponda'ry publishers (in their preparation of 
indexin^abstracting announcement tools in the form of 
printed Ipubiications and/or connputer*readabIe /databases, 
for distribution to the public); , ^ ' 

• research institutes (in their analysis of technical literature 

and preparation of reports synthesizing their findings); and ^ 

• ()rimary publishers (in their creation, publication, and 
distribution of nev/, original literature, e.g., handbooks, 
critical reviews, and Bummaries). ? 

Some clearinghouses are designed to function in some ways like all of these types of 
^enterprises, and others are designed to 'function like only one or two* 

Clearinghouses that are involved in all or nearly all of the functions shown in ' 
the conceptual model can be considered fuU-service or full-capacity organizations* 
Other clearinghouses work with other organizations that perform additional roles in y 
the information utilization process. For example, in the field of education, the 
ERIC clearinghouses are involvecT^pnoriari^ in acquisition and storage (Step 3), 
organization and control of the literature (S^ep-4jL,,^^d generation of information 
synthesis and analysis products (Step 5). Through such ihteirnediary users of y the 
ERIC database and microfiche collection as state and local education information 
centers and university libraries, ERIC information is made widely available to 
education practitioners, administrators, and researchers. j ' * 

Another model exists in sopne of the health programs where a fully integrated 
program is designed to support the information utilization process. Examples 
include the National Clearinghouse for Mental Health Information, the National 
Clearinghouse for Alcohol Information, and the High Blood Pressure Information 
Center. In these areas, it is difficult to define the boundaries of the "clearinghouse'' 
because it is embedded within a larger organization that performs many of the 
translation and dissemination ^tJ^nction^ "In some cases, clearinghouses in these 
program ajreas bear a close resemblance to special libraries in other cases, they 
,cover a wide spectrum of activities and are, in effect, a total program. 

Other clearinQhouses, such as Project Share, are not tied to a specific program 
and do not have a'^uilt-in support network. They perform a brpad array of functions ^ 
themselves. They publish the results of research in their fielc^ acquire and store 
literature (particularly unpublished literature); develop a database for retrieval of 
relevant information as needed to respond to request^ and develop information 

. , 13 



synthesis publications to translate findings- into materials that can be used by their 
^target audiences* Where no other linkage organizations exist, clearinghouses also 
take on some ^esponsibility .for establishing user networks, such as in the case of the 
Consumer Education Resource Network* 

The functions assumed by the clearinghouse can be seen to depend on: 

• the nature of the mandate that has led to the establishment 
of the clearinghouse; . _ ^ ^ ^ 

• * the existence of other infbrmation service organizations, in 

the public or private ^ sectors that already adequately 
perform some of the functions necessary to serve the 
clearinghouse's .target clientele and 

^ • the degree to which the clearinghouse can tap into, rely on, 
or otherwise take advantage of existing information services 
related to its mission* 

The concept of the functions a clearin^ouse should perform is constantly 
evolving* In 197^, Herman Weisman developed a definition that suggested that the 
major functions of a c'aaringhouse were document collection, document processing 
(e.g*, indexing), document storage, and document publication or reproduction. 
Issuing newsletters, developing dii^tories, and issuing accession lists were 
considered to be "minor** activities* The^^eparation of critical reviews and 
^ state*of-the*art reports were not considered clearinghouse functions at the time 
(Weisman, 1972:20). ' 

Since that' time 'Jthe'' announcement, repackaging, synthesis, and networking 
functigns have become^ increasingly important aspects of many clearinghouse 
operations* Several factors have contributed to thfs expansion in the definitidn of a 
clearinghouse's functions* The recognition that dissemination alone does not 

produce behaVioral change may become a powerful motivating factor ir> linking up 

\ ^ 
with organizations that perform the additional steps in the research utilization 

^ "> 

process or in taking on those functions within the clearinghouse* The primary reason 
for such a change is that the environment in which the/ operate (i*e*, the social or 
economic problems that need to be addressed, the subject matter and information 
involved, and the clearinghouse clientele) is changing and clearinghouses must adapt 
to these new situations and demands* 

Clearinghouses als^ change as they begin to fulfill their initial objective^ their 
success generates a need for new objectives to be met* • For example, a 
clearinghouse may be established to bring under centralized, bibliographic contr(!il 

1.4 
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all of the literature of a particular type in a specific area« As that literature is 
brought under controli the clearinghouse may perceive a need to broaden its 
coverage in related subjects or in additional types of source materials, or to 
institute new products or services from that database, to serve its users better and 
to attract the norvusers in its target population* The expansion into new activities 
is a response to the 'desire to serve the clientele better; however, it brings with it 
the potential for encroachment on the work^being performed by other information 
service organizations, (The problem of overlap and duplication among information 
services is discussed in Chapter k.) ^ 

\ These two important characteristics of clearinghouses— their variety and their 

need for change-*should be kept in mind while reading this report. One should not 

< 

imagine a single' clearinghouse model or any final definition 'of clearinghouse 
objectFves and functions* If all clearinghouses were' functionally identical, they 
could not be responsive to the different missions, objectives, clientele, and service 
Deeds for which individual clearinghouses have been established. If the 
clearinghouses were not prepared, to evolve as their information environment 
evolves, they would likely become obsolete. 

The need to adapt and respond to an ever chan^in^nvironment is likely to take 
on added importance in the early I980's, because of changing fiscal policies in the 
Federal government. • The perceived need for govefnment-sponsored information 
dissemination services*-and for clearinghouses as a me^ns to (Provide those 
services-^is not likely tb disappear during the forthcoming ^eciod of re^prioritizing 
Federal spending. However, the role and effectiveness of clearinghouses in 
achieving disseminatioi objectives are likely to be scrutinized even more closely . in 
this new fiscal environment. An Understanding of the role clearinghouses can play in * 
information dissemination is, therefore, critical to establishing policy directions for 
the coming years. This role is discussed in the ne)?t section. ^ 

• # 

Clearinghouse Dissemination Objectives 

The assessment af clearinghouse eff.ectiveness must be madfe in context of a- 
basic understanding of their objectives. It is generally agreed that the dissemination 
of information— to inform a target audience about the availability of information on 
a' given topic and/or to provide that information diredtly— involves a set of 



.objectives that are quite distinct from those that seek to bring about behavioral 
changei to encourage adoption of specific innovations and/dr practices, and to bring 
about greater research utilization. This distinction is reflected in the definitions 
given by Blake ( 197305-4): . , 

• dissemlnatiorv-the act of creating an awareness of or an j 
interest in a practice considered to be a worthy solution 

to a need or problem among potential users, e.g., the 
^ production and distribution of printed or audio-visual ^ 
materials, conferences^ interpersonal communication, 
traveling seminars, answering queries, oral presentations; 

• diffusion— the process by which a practice or solution is 
moved from the producers and cjevelopers to consumers' 
who 3d|et and translate into practice, e.g., 
awarenf^ilTinterest activities, visitation/demonstratidn, 
t^chnidal assistance for in^fallation, staff training in 
preparation for field testing, evaluation to determine 

^ impact. 

The distinction between dissemination and diffusion .is particularly important in 
establishing a realistic role for information plearinghouses. If dissemination is the 
goal of government information services. thea,axB e_ctation s for evaluating their 
effectiveness should be stated in terms of increasing the- ^target^ audience's 
awareness of and interest in specific practices or findiogsT On thorother hand, if the. 
goal is diffusion and adoption, then performance .measured, should^^stated in terms 
of the type of change to be, effected. What must be realized,^ however, Chat 
methods suitable for achieving dissemination are not necessarily likely td* produce 
diffusion. 

Recent studies indicate that the techniques most useful in pro^iucing effective 
diffusion and research utilization are significantly different from the dissemination 
.techniques used by clearinghouses. Strategies for influencing the populace or 
certain segments of the^^opulace involve use of a wide variety of information 
exchange techniques and channels. For example, to encourage acceptance and 
widespread implementation of new practices and ideas among practitioners is likely 
to require active face»to-face communication between the designer's of innovations 
and the target practitioner group, to facilit ate tb e practitioners' particifsation in the 
innovation adoption and implementatiprfprocess (Stevens and Tornatzky, 1980:34). 
Yin and Gwaltney ( 198 1:569-57 15 also suggest the use of linkage agents and user 
panels as methods for facilitating the necessary communications between 
practitioners and researchers. - 
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These studies also suggest that research reports and written materials are only 
one source of information used by practitioners* Other sources include first-hand 
experi^ce, the experiences of trusted colleagues) and information obtained at 
conferences and workshops. Frequently these other sources are considered more 
reliable than the written word, in part because the user can determine whether the 
conditions under which the innovation was tried resemble the conditions in his or her 
environment (Yin and Gwaltney, 1981:559; Weiss, 1980:390). Exogenous factors, such 
as an organization's willingness and ability to change, also play a part in determininfg 
whether an innovation will be adopted (Knott and Wildavsky, 1980:545). 

The role assigned to clearinghouses typipally (although not'in all cases) stops 
short of these types of intervention processes* Clearinghouses are , 'activa 
participants in the information transfer process but gejierally only supportive 
participants in the utilization process* Their' support is to various types of linkage 
agents in a given field and/or to specific program areas within their soonsoring 
agencies. Therefore, unless a cfearinghouse is given or assumes an active role in 
bringing about change^ it cannot and should not be held accountable, after the fact, 
for accomplishing diffusion^rdlated types of objectives. 

' The research .summarized in this chapter has important implications for 
determining future policies for th^e planning, ^operation, and evaluation of 

' clear inighouses. The criticism of the failure* of clearinghouses .to produce 
information that affpcts decision-making stems from unrealistic expectations about 

* what effects dissemination of research findings can have on social policy and 
pn^gramming and from inadequate understand^.ng of the interventions needed to 
produce behavidral change. Recognizing what informatioTi dissemination and 
clearinghouses can accomplish permits policymakers to: 

. li establish more realistic objectives for clearinghouses;,^ 

\- 

• ' plan the structure and placement of clearinghouses so that 

they are part of a' support, network that establishes the 
cocnmunication needed to ensure adoption of proven 
innovations, if that is the goal; and ^ ^ 

• ^ •estab'lish realistic standards for evaluating the performance 

of clearinghouses.'' 
' - . o ' . 

Policymakers must decide whether it is , appropriate for the government to be 
funding, and operating these types of facilities. 
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PLANNING CONSIDERATIONS AND ISSUES' ^ 

Introduction * • \ * \ ' , 

Planning is the. name of the process through, which the goals of an enterprise are, 

established and/or translated into operational policies bnd procedures* Every 

*• • • . 

economic and social unit in both the public and privatejseetors engages in planning, 
sometimes in an intensive and highly explicipwaybut often in a more casual and 
implicit way* ^ There is a moderate^rr^Ia^on^twee^^ quality of planning and 
the success of the operation^ifig pl'annedr An enterprise may succeed in spite of 
poor planning if it embo0i€s a very sound or unique idea* On the other han.d^ it may 
f^l in spite of good planning if the idea or concept behind the plan is too fragile, the 
circumstances on wf)icn thp<plan was based change drastically, or the plan, onpe 

developed, is not fbijiowea* 

y - " ^ , ^ * 

This chapter addresses the /ormal planning process as it relates to the 
establishrrient^^nd/Operation of information clearinghouses. The chaptW draws upon, 
and e^uA^jp^uqh further interprefation, the f indings^ from puc ^uay. of, about 30 
cl^inghouses and other types of information-dissemination organizations. The 
chapter is not designed as a prescriptive handbook for planning any given 
clearinghouse.' As becomes more evident in the discussion, the issues and choices 
involved in such' planning .ar6 complex, and there is a dearth of w^U-tested pi^mciples 
and easy answers. Therefore we attempt primarily to. illuminate the major issues 
and choices as 'they relate to the establishment of nejv clearinghouses and to 
changes to e^cisting ones. Consideration or the latter reflects the fact that planning 
is not a one-time, qnce-and-for-all type of activity. Rather, it represent&^^isrshould 
reppc sent— an ongoing activity in the / management process. Ipiz cpapter will 
address the topic of planning as it relatei^o: 
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• clearinghouse missions, objectives, and functions; 

• management and <^ganizational considerationjs; and 

• operational Considerations* * ' . 

Clearinghouse Missions, Objectives, and Functions , 

In the description of any type of system or service, it is usjfut^txj-dtffeTeritiate 

three levels of specificatiori. The "mission' (or^ sometime^ "goal") is typically a 

broad statement of the basic purposejJDderlyTng the establishment of the system^in 

this case,"the^nfornrtation clearinghouse. Because the generality of most mission 

statements, the re^ader cannot determine from them what functions a clearinghouse 

actually performs or how it performs them. ^ 

\ * ^ . ^ ... ' 

Statements of "objectives" translate mission statements into more specific 

terrfis,.that can help the planners to begin defining clearinghouse functions— the 

latter representing a still greater level of specificity. A typical hierarchy might be 

as follows . * 

• * Mission: disseminate safety information; ^ 

• ' Objective, (one of several): identify numbers of organizations 

^rid indivldi^ij^^who need safety information; and 

• . Function/actfvjty (one of many): screen all (or some 
percentageXof the published professional literature) as well as 
unpublished reports and dissertations, for iniTormation relating 
^ to the* topic of safety. 

Onfii of the major f indirtgs of the earlier phases of this study was the (general 
lack' of specificity in clearinghouse objectives and the lack of articulation of the 
linkages between the mission and>^ objectives of individual clearinghouses and 
between objectives and ^their functions, or activities. ' For the most part, 
clearif^ghouses'are thought of^ and defined in terms of « the .activities that they 
perform, not in terms of what those activities are expected, to accomplish, (i.e., 
^ their objectives). 

It is taken as an article of faith that information (Jissemination is a service for 
public benefit. ThereforfeV-clearinghouse objectives aref- rarely stated in terms of 
.expectations for Outcomes of a spe.cified nature .(e;g., tne adpption of a particular 
innovation, improy^d^practice, or Improvement in a particular^ social cortdition). 
Ins'tead, they, are phrased^'in terms of-^inf6rmatiorvre|afed activities (e.g., to help a 
certain population stay abreaj^t of current resee^rch or practice, or to make a body of 
literature more readily availableJn* a certain area, or to distill and. otherwise m^j; 
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. sen3e ouLof the growing' body of knowledge in field or problem area). As one 
program director put it, "Infqrmation dissemination > alone cannot bring about 
• change, but change is\not likely to ocQur^ without informatfion." \ 

The initial expression of need for a clearinghouse • to perform information 
dissennination activitiesAcan stem from any of a number o^ sour9eSy for example, 
through legislation or agencypribritiesy as a result of constituency concern^/ or as 
' an integral part of* the conceptualization vf solutions for a particular problem. 
Regar,dl63s of the impetus for clearinghouse development, there must be a realistiii 
^and speprfic statement of mission snd Qbjectiyes to facilitate the identification and 
development of an appropriate, set of informatiop seryice activities for the 
clearinghouse. - ' ^ 

(. 

Without a clear and unambiguous mission statement, and without a reasonably 
specif ic definition of objectives, one cannot: 

' id establish realistic expections for the clearinghouse; 

• provide a basis for designing the policies and procedures of the 
clearinghouse; or , ^ <c 

• provide a basis for determining wheth^f^or how well a 
clearinghouse is doing^ its job. « . 

There is a saying that "If you don't know where you're going, any road will get 
you there." What it means in' the clearingViouse context is that if no objectives or 
targets have been specified for the .clearinghouse,; almost any course of action qr 
any combination of clearinghouse activities is equally valid and ^equally defensible. 
The implications for the evaluation .of clearinghouse activities are profound. We 
will return to this problem in Chapter 6. For purposes of this chapter, we will 
assume recognition and acceptance of the fDremise that clearinghouse spohsors-and 
planners want Co be able to answer questions such as these: 

• . What kind of information is to be.dissemirlated? 

I' " 

^ V ' • To whom will the information be disseminated, and if others 
are intended to be the ultimate beneficiaries, who are they? 
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What is expected to be gained by the dissemination activities? 
How is the dissemination to be accomplisheci? 
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The choices available in answering these questions must necessarily be. studied 
-together :in_.the .contex tx.f JechDical, economic, '-^.nd political realities, and in the 
context of the original expression of need. Decisions in one area influence and are 
influenced by decisions in other areas, and all of them must take realistic account of 
available resources and constraints* 

A context for addressing these questions was provided earlier in Chapter 1 and a 
more specific context is provided in the following two subsections* 

* Dimensions of Information Dissemination " ^ - 

Information dissemination has a number of dimensions that need to be defined 
in the process of ''specifying the objectives of a clearinghouse. Among the most 
impbrtai^ are the characteristics'of thet "information package*' being disseminated, 
the degree of neutrality (as opposed to advocacy) reflected in the information that 
is being disseminated, and the^^aggressiveness .with which Jthe information is being 
disseminated* . ' * ' 

The Information Package . For any giVeri clearinghouse^ a multiplicity of 
outputs, or information packages, may be needed or desired* Those packages may be 
formal publications * (e*g*, state-of-the-art reports and critical summaries), 
informally prepared and bound materials (e*g*, bibliographies preselected topics), or 
responses to inquiries that might include a cover letter and a bibliography or listing- 
of referral' sources. Since resources are typically scarce and must, therefore, be 

allocated carefully, it is necessary .to make hard choices and establish priorities 

^ • - . ' ' 4- 

among various types of information packages. The clearinghouse can, for example, 

focus its dissemination effort on informing a target audience about the availability 

of information (and the means for obtaining the information) on given topics. On 

the other • hand, the clearinghouse can focus' on ^disseminating. 

knowledgen -information content— to educate the target audience to the findings 

from research and practice. These tv.o objectives, although not mutually exclusive, 

call for very different kinds of information packages* A clearinghouse may try to 

accomplish both objectives. However, it should be aware that each of these 

objectives ^has its own s^ of requirements for support activities, for information 

formats, for appropriate channels of dissemination, and for staffing. ^ 



. .... 



The foregoing discussion addresses the nature of th^^ inforijiation package. An 
-equally— iiViportant and-related "Consideration's the tyfae of information being 
disseminated. '^The focus of most clearinghouses! is on bibliographic 
informatiorH*information packaged primarily in printed fornr\ to refer recipients to a 
'Sec of source materials that^are also packaged 4n printed ^prm. Similar reference 
activities and products can be envisioned^or numeric or statistical data» referral' 
sources (e.g^ programs^ people, or organizations), or for norvprinfc materials, all of 
which tend to be much more "fugitive" than the printed record c^f knowledge. 

Neutrality versus Advocacy ^- The neutrality-versus-advocacy issue has to do 
with the viewpo'^int tHat is reflected in the clearinghouse's disseminatjon efforts. 
Policies must'be articulated in terms of whetherte^resent the^target audience with 
inforniation about materials on all sides of an issue or^only about chose that support 
a particular viewpoint. \ 

A number of clearinghouses today reflect current public policy\on q given topic 
(e.g., the hazards of smoking) and by their very existence imply an advocacy 
positioa Although some of these clearinghouses are relatively neutral in their 
, coliedtion of information and in their responses to requests for references "on a given 
topic, theii^ information analysis publications more clearly reflect their advocacy 
position. o , 

Although the functions of a clearinghouse with a particular advocacy position 

may not differ signifiqantly from those of a more neutral clearinghouse, there are 

major differences in their guiding principles and objectives. It is one thing to 

develop a strategy to inform the public about the issues in matters* of public policy; 

it is quite another to tJeji/eiop a strategy for disseminating a particuW viewpoint or 

message about those issues. In translating legislative or agency intenf into a set of 

objectives, the appropriateness of the clearinghouse role ^ with regard to the 

'I "* 
advocacy/neutrality issue must be fully explored and made explicit. 

Aggressive ^Versus Faosive Dissemination . A -third important dimension of 
information dissemination has to do with the fundamental meaning of the concept 
"dissemination." Disseminatioh can be accomplished through an active and 
aggressive distribution of infornnajtion— focusing on the "exposure"— or it can be 



accomplished through more passive and responsive distribution of information, on 
request*«focusing on availability and access. Unless further specified^ dissemination 
can be takeiV to mean any of severaLvery different kinds of activities, as suggested 
in Exhibit lA^ where three modes of dissemination are arrayed against three types 
of information^ ^ * 

'A number of operational implications ^exist for each of the nine cells in the 
Exhibit, particularly in the area . of outreach and marketing. If, for example, a 
clearinghouse intends to distribute copies of a particular publication to targeted 
individuals .who have not requested that publication, a procedure must be developed 
to provide assurance that the publication will be of value to them. In this area of 
direct distribution, versus responding to requests for publications, clearinghouse 
roles and cost^reco^jsry philosophy become most critical. 

The distinction thiil is raised in this Exhibit between requesters and recipients 
involves some complex problems that must be faced in defining a .clearinghouse's 
target audience and users. 'This is the topic of the next dubsection. 

Target Audiences and Users 

^The majority of clearinghouses in our study identified one or more sets of 
target audiences* These target audiences include professionals (e.g., researchers, 
practitionerst and administrators), organizations (e.g., foundations, associations, 
agencies, and other information services^ and the general public or special 
population groups (e.g., handicapped individuals, low* income persons, and individuals 
with a particular health problem). The specific occupational or personal 
characteristics of these target audiences are defined in context of the mission of 
the particular clearinghouse (e.g., an educ^tion-related clearinghouse may define a 
practitioner as an elementary sbhool teacher and a health-related clearinghouse may 
define a practitioner as a physician. • ; 

In practice, many clearinghouses serve a broader range of individuals and 
organizations than the defined target audience; these individuals become actual 
users through their requests for information and publications. However, not all 
clearinghouiies are this accommodating. Some follow a policy of referring 
non-target audience requesters to other organizations better equipped to handle 
their requests. Although a clearinghouse may elect to serve ail who request 



EXHIBIT 2.1: COMBINATIONS OF DJSSEKdlNATION MOOES AND INFORMATION TYPES 



>sType8 of Information 
Disseminated 

Dissemination ^^^^ 
Modes 


Secondary 
(Abstracting 
and Indexing) 
Publications 


Documents that 
Have Been 

Announced in 
Secondary • 

Publications 


Original 
Publications 
Developed by the 
Clearinghouse 


• 

Distribute only on ' 
request 








' Distribute to those 
who subscribe or 
otherwise request to 
receive automatically 




• 




Distribute to selected 
individuals and' 
organizations in a • 
target population with* 

' out any prior knowledge 
that they are interested 
in being a recipient 

ft. > 
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information, it is nevertheless important that its efforts be given some direction, as 
well a3 some boundaries, through the identification of particular targets* The 
characteristics of that audience (including who they are, where they are located, 
how many there are, wh&t their information-seeking and information*u3ing behaviors 
are) are the single most important factor In determining the,shape*pf clearinghouse 
services and publications* 

Having emphasize^ the importance of target audiences to the definition of a 
clearinghouse, we should point out with equal emphasis that the answers to some 
questions about the characteristics and needs of thosi^ audiences may not be 
knowabIe*«or at least not readily j<nQwable* Defining a target audience is often 
much ^more difficult than it is imagined to be. Those directly involved in 
clearinghouse programs understand well both thei difficulty and the importance of 
distinguishing between the characteristics of practitioners and those of researchers, 
or between individuals in their professional/occupational roles and their role as 
citizens, in their development of appropriate publications, services, and channels of 
dissemination. The' following questions help to illustrate the difficulties that can be 
encountered by clearinghouse personnel in translating the spirit of a legislative or 
agency policy into a realistic statement of intent on who is to be served* 

• Is the target audience actually definable at any given point in 
time (e.g., practitioners with a particular health care delivery 
responsibility) or do individuals become a part of that audience 

\ ortly under special conditions or when confronted with a 

particular problem (e.g., pregnant teenagers)? 

• Is the target audience direcCly reachable, or is it necessary or 
desirable to reach its members through a set of intermediaries 
(e.g., private organizations and/or state and local agsncies) 

that are*likely tc have contact with that audience? ' . 

/ 

* Answers to these types of questions are needed to formulate a successful 
outreach strategy and to determine the types of linkages that must be established to 
reach effectively (and cost-effectively) the ultimate beneficiaries of clearinghouse 
^services. Therefore, although it may be politically desirable to speak^ in 'terms of 
H ultimate beneficiaries as the target audiences for clearinghouse publications and 
^services, it is also of practical importance to clearinghouse* personnel' to identify the 
actual audiences, including intermediaries, with whom they will be xxj direct contact. 
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Management Considerations and Issues 

This- section uses the major findings from our data collection to illustrate the 
ways in which certain ^variables can influence the design of a clearinghouse and/ or 
aid in the management of an existing clearinghouse* Major decision points and 
issues are discussed in t^fe areas of budgeting and f inane i^i^planning, organizatio'hal 
BT}^ staffing options, the role of automation, and marketing. - 

Budgeting and Financial Planning - 

Final formulation -bf a clearinghouse design, including its activities, staffing,- 
and service objectives, is clearly dependent on the budget that is available to 
develop and operate the clearingnouse* Therefore the clearinghouse design should 
be conceptualized in different configurations to fit realistic "minimum" and "ideal" 
levels of funding* 

The mean funding level of 19 Federally funded clearinghouses, including those 
operated internally and those under contract, is now about $1 million. The range is 
between $108 thousand and $4.8 million. The contrast between these two extremes 
may be somewhat exaggerated, because the smaller budget figure is associated with 
a Federally operated clearinghouse and all of its direct apd indirect costs may not 
be represented. Nevertheless, the differences in budgets between these two 
clearinghouses reflect significant differences in their target audience. In the case 
of the smallest clearinghouse, the target audience is a very restricted and 
well-defined set of professionals; in the case of the largest clearinghouse, the target 
audience is the "general public." In the case, of the smaller clearinghouse, the 
processing of information into a collection is a major objective in the case of the 
largest clearinghouse, a' major objective is to be able to handle some 240,000 
requests per year. 

Some other key budget-related findings from our earlier analyses are presented 
in Exhibit 2.2, Because clearinghouses are labor-intensive enterprises, it is not 
surprising that the variable of "staff size" shows the closest relationship to the size 
of the clearinghouse budget. Larger budgets and, in turn, increased staffing per^nrHt 
a clearinghouse to engage in additional activities and to -handle higher volumes 
Within those activities. < ^ ^ 
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EXHIBIT 2.2: CLEARINCSHOUSE CHARACTERISTICS. . BY , SIZE OF THE 



OPERATING BUDGET 





Size of Operating Budget a/ 


Characteristic 

■ 


$249,999 
or less 
per year 

(n=r6) 


$250,000 - 
749,999 
per year 
(n s 7) 


$750,000+ 
per year 

, (n = 8) 


Mean Size of Staff (FTEs) - 




13.6 


33.1, 


Mean Size of Collection 


10,103 


11,985 




Mean Number of Items 
Processed in Most Recent 
FY 


2,929 


2,621 


. 7,695 


Mean Number of Requests 
Processed in Most Recent 

FY ; ' * 


6,550 


25,102 


* 95,674 


Mean Size of Mailing List 


■ 6,877 


4,433 


50,929 


Mean Number of Publication 
• Series Issued in Most 
Recent FY 


4.1 • 


5.7 


6.6 



S'Oata are ceported for a total of'21 clearinghouses that provided the 

budgetary data needed to classify the organizations by size of budget. NTIS is 
excluded because the volume of its activity would skew the means in the 
$750,000+ category. . 
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Clearinghouse Objectives and Economic Realities^ In conceptualizing various 
clearinghouse configurations, one must keep in mind the relative costs of various 
activities* For example, user services is typically one of the most, if not the most,, 
labocrintensive activities. It tends to require a significant pe. :enta^e (a mean of 24 
percent in c3ur survey) pf all operating costs (Applied Man^a^n^nt Sciences, Inc. and 
Cuddra Associates, Inc., 1981:7.15). That is not to say," however, that the 
publications or database development acEivities could not equal or surp'ass these 
median percentages for any given clearinghouse budget. The ma^n differences 
among the three major types of .clearinghouse activities is that costs in the other 
two areas are mo^a- easily controlled (e.g., through the specification of numbers of 
publications and their associated budgets and through the ?pecificati6h of numbe 
of documents to be processed). 

The amount of time spent by staff in handling reference and referral questions 
^ in^a user services activity can be reduced through management policies that limit 
the levels of service that are.to be provide^j, and through the use of computer-ba$ed 
literature searching services. However, there are no easy * ways to controh labor 
costs in situations with high or highly variable volumes of requests. The knowledge 
that user services is likely to be a significant operating expense means that in 
. planning, careful consideration should be given to making, choices such as the 
following. Given^e objectives of the clearinghouse, is the provision of reference 
and referral services as necessary as some oth^ activitie^s, such as the development 
of the database or analysis of the information for preparation in tet-packaged form? 

• If the answer is YES, can other activities be given-less priprity 

• or b;e excluded entirely ^n the clearinghouse design? % 

• If the answer, is NO, can, the same amount of money, or less 
mbn.ey, be spent initially to establish linkages with existing 
organizations (e.g., libraries and other service units within 
universities, hospitals, associations, or social service agencies 
^and organizations) that can provide some or ail of theie user 
services? ° Or can limits be established on who will be served 
or on what tyf^es of requests will be handled, anckcan those 
limits really be observed in practice? ' * 
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Although there may be no completely satisfactory answer to any of these 
questions, the U'ade-offs are clear and so are the options: one can provide no 
.service ^t all; one can provi6e only a limited range of service;^ or one can attempt 
to provide a full range of services, knowing that they/may lack timelinessi quality, 
or^other generally desirable characteristics* 

. This discussion introduces a dilemma in clearinghouse operations that was 
revealed in interviews with 27 clearinghouses and is represented .in the following 
contradiction: clearinghouses are^-^iven the mandate to disseminate information 
(e«g«, by responding'^ta^quests that require individualized handling), but many of 
the clearinghouses cannot actively promote this service, because they have neither 
the capacity 'co grow nor the budget to handle the higher volume of requests that 
might be generated. This dilemma must eventually be resolved through: (1) 
cost-recovery policies (see Chapter 3); (2) the establishment of limitations or^ 
reductions in expectations /or the dissemination activities in which a clearinghouse 
is. to be involved^ or (3) the development of creative contractmg practices that both 
permit and provide incentive for clearinghouses tq^gro.w and to be responsive. 

Careful planning and budgeting in the design ^stage are needed to reflect 
priorities among^the clearinghouse activities being developed. The management of 

the budget requires good cost-control procedures, the topic of the next subsection. 

\ 

Cost-Accounting Expectations and Practices. Our data^llect^oQ suggests that 
with a few major exceptions, little . attention has been given to establishing 
cost-accounting practices and thfe^ appropriate management information systems 
that support sound ongoing financial planning. These systems are needed to develop 
realistic expectations within an agency for what it costs a clearinghouse to be "in 
business" and are especially important if cost recovery is envisioned. The problem is 
particularly acute in Federally operated clearinghouses, the sponsors of which 
sometimes have no data on total operating costs. Although contract costs are 
known for tht3se clearinghouses operated by .contractors, the data available are 
typically not organized in such a way as to permit the computation of per-unit costs 
that are meaningful outside an individual clearinghouse. 
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If sound financial planning for agency-wide information management is to 

occuri some reasonably uniform means for expressing costs must be de^jeloped. Sucfh 

"standard??' need to be framed as guidelines, to accommodate major differences 

among clearinghouses*. However, development of the guidelines will not be easy. A 

major difficulty has to* do with the interrelatedness of many ,of the typical 

clearinghouse activities. One activity may produce outputs that support several 

other activities, making it difficult to attribute an accurate share. of the costs to 

^eacH of t^e activities being supported. For example, a clearinghouse^devefoped 

V 

conrrputer-readable database may be used in both the preparation and publication of 

» * 
"spin off bibliographies and in the provision of literature searches* Some way needs 

to be found to allocate the cost of the database between those two activities. 

Significant progress in measuring cost-*effectivenes8 is not likely to occur uritil some 

reasonable true costs can be compared against some standard costs or range of costs. 

* s 

Organizational and Staffing Options . , 

Organizatiorvrelated areas in which alternatives need to be weighed include the 
selection of operating organizations, the placement of a clearinghouse within a total 
program, the de^jj^tioh of functiorfal work units or divisions, and the establishment 
of staffing requirements. 

' Selection of Operating Organizations. Among the 24 Federally supported 
clearinghouses studied, about half are operated entirely under contract. The others 
"^are* managed and operated within government agencies, although in a few gases, 
selected functions are performed under contrast. The "in-house-versus-contractor" 
choice may not bh available if, for example, the Federal staff positions necessary to 

3 

operate the clearinghouse have not been authorized. However, if the option exists, 
the balance of performance ipd benefits needs to be weighed for in-house versus 
contractor^operated clearinghouses in these areas: 

• ^ (lower) costs; 

• , cost control , * o ^ 

overall performance in outreach/ visibility; 

• constituency developrnent; 

• continuity over fiscal years; * ^ 

. • adaptability/flexibility t;o meet changing needs of populations ^ 
and programs; 

^- . - 31 • ■ . 

2.13 



• • adaptability/flexibility to stay* abreast of and have access to < 

^ the newest equipment/ procedures, techniques, etc^ . o * ' 

• ' ability to, rneet staffing requirements; - *^• 

• service riasponsiveness; and , • 

• • ability to integrate into total agency program effort.^ 

^Tl^e^onsensus among^several- clearinghouse operators apd Project Officers with 
sihom discussions have been held is that the balance of benefits or performance 
.typically re^ts on the side of ccntractor*operated clearinghouses* It should be 
no^ed, however, that in many of these areas, including costs, the 'basis of the- 
judgments is largely cohje^tural because comparable data are not available. 

Organizational Plapement of the Clearinghouse * A Federal agency^ can find 
itself subjected^ to the criticism that its management is not aware of the 
clearinghouses that its funds are supporting. This should npt be too surprising, given 
'the fact that not all organizations classified in this study are known, or readily 
identifiable, ^s clearinghouses* Clearinghouse-like functions can be, and sometimes 
ar«, embedded in a total program area, with little or« no separate identity and 
visibility* For example, a library collection can be established initially to support a 
particular program or, research area,^ and as the need for dissemination about the 
program or' about the' problem area in which it is working grows, the library can 
evolve into a rather different type of en^ty without changing its name or external 
image* " 



To accommodate this ^phenomenon, a distinction . can be made between 
"integrated" ("programmatic") clearinghouses and "independent" cle^^ringhouses* The 
dichotomy is not altogether satisfactory, but it permits useful discussipn of 
differences in organizational structure that clearly affect the way in which. a 
clearinghouse is designed and the wa;^ it is perceived .within its parent or sponsoring 
agency* ^ ^ ' ^ 

For example, the ERIC system, although a part of the total information 
dissemination program in NIE, operates essentially independently of other program 
efforts, such as those in the Research and Development Exchange (RDx) and 
National Diffusion Network, that are directed toward friformation utilization* The 
Project Share and the Arthritis Information Clearinghouse also illustrate the 
concept of an "independent" clearinghouse. In contrast, however, the clearinghouses 



or clearinghouse*lik^,components in several NIH programs and others (e.g»» mental 
healthi high blood pressure) child abuse) are much more integrated into the program 
areas and into fhe overall agency mission* 

Althdugh one cannot conclude that organizational placement is the mi5st 
significant^ variable in accounting for differences in ways clearinghouses are defined 
(i*e#, m the functions they perform and the objectives they work toward), it is clear 
that sueh placement is one of the most important variables* Integrated 
clearinghouses are more Uleely functionary to resemble special libraries and to have 
a* more limited range of responsibilitiesi/particularly in informatibn analysis and in 
' outreach/promotion. In the health area particularly, ,it appeers that a set of 
social/research/information aims have been addressed .through large^ multi* function 
program units, which are. difficult, to compare with clearinghouses whose aims are , 
primarily information-related. ^ « ^ " 

/ Definition df Functional Work Units. Although considerations in this area may 
be viewed as matters of manac^ement detail, differences in the ways clearinghbuses • 
are structured functionally and in the ways that staff are assigned to those 
functions, contribute' significantly to difficulties in defining and implementing 
standard cost- accounting procedures. In some of the larger , clearinghouses, 
organizational charts reflect distinct work units and the assignment of staff to those 
different units.' However, in the smaller clearinghouses, particularly those with a% 
strong service orientation, the work units are not distinct and 'the staff are 
> perceived as team members who "pitch irf*- to help wherever the need exists. 

For clarity in reporting the various activities of clearinghouses, a single 
'taxonomy was used to analyze the data we collected. This classification of, 
. activities includes; " . 

' • The input and processing function 

information resources (i.e., collection development) 

processing of materials and facilitation of bibliographic ^ 
access; 

e The informa^:ion anal/sis function 

pleannghouse publication^ ^ ^ ' 

• The dissemination 4nd user services function t- » 

user services 

promotional and outreach activities; and ^ 

• Clearinghouse management. 

' ' , 33 • ' 
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Other equally valid classification schemes or organizing principles can be 
developed on the basis of staffing and' on the basis of the kind of interaction among 
" activities that needs to 'be facilitatad. For example, tiser services personnel are' 
likoly to be in^ the best position to detect the need for a topical bibliography or 
^particular ^ inforrnation analysts publication. Thus w|jile user'- services and 
putnlcaitiofls'^Sevelopment may be treated conceptually as separate work units, the 
personnel in acsmalter clearinghouse may actually, be assigned to help in both units* 
In larger clearinghouses, an organizational system must be consciously developed to _ 
provide appropriate levels of cbmmunication and feedback among the different 3'ets 
\ of personnel assigned to different fungtional units/ 
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^ One example of the complexities^ in defiiiing and implementing cost^accounting 
procedures^ that, results from the interrtflatedness of clearinghouse activities is 
represented in the conceptual versus actual organizational ' placement of . 
announcement publications* These, indexes or index/abstract journals are the end 
product of a series of information^ input processing steps that produce a master 
database, although*' their production may organizationally be a part of the 
publications preparation work unit. Regardless th^/r placement, accounting 
A"ptirlclples must be established that treat these and other products derived from the 
"master*' database in one of two ways: (1) they will share the "first-copy" costs 
associated with the development of the full database, or (2) t^ey vyill be treated as 
by-products that bear .only the incremental costs af^ocifited with their unique labor, 
prmtlng,^ distribution, and other direct costs. This choice becomes ..particularly 
relevant to our discussions of cost recovery, in Chapter 3; 

o 

I 

f 

> ' Staffing Requirements. The knowledge^ and skills required in a clearinghouse 
operation, particularly a fpll service clearinghduse, span a number of disciplines.-* 
Some, such as public administration, commi/nications and journalism, and library and 
information science, are process-related disciplines. Others, such as in the medical, 
social science, or education areas^ are content-related disciplines that are relevant 
to the clearinghouse mission. Most of the clearinghouses^we studied have a majori^.y 
of staff members with content-related backgrounds. 
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There is, of course, . no formula that can * be. prescribed for achieving an 
appropriate mix of backgrounds. If a full service clearinghouse is the objective, 
subject expertise is required^ not only for information analysis and synthesis but also 
to provide credibility for the publications and services. On the othar hand, infusion 
ef process* oriented skills is also vital, to help ensure an efficient implementation of 
information processing principles and techniques (with a minimum of re^Jnvented 
wheels) ^nd that professional skill and understan/iing are brought to the ^task of 
analyzing user needs and selecting appropriate chajhnels for information 
dissemination. * . - J t 

Because clearingjiouses are functionally hybrid organizations, the requirement 
for an appropriate balance of educational backgrounifs is only one element of the 
broader need for appropriately balanced staff capabilities. A plan for staff 
development is also needed to^bring about a blending of those multiple backgrounds 
and experiences to create a cohesive and mutually supportive clearinghouse staff. 

The Role, of Automation 

Computer and communications technologies are being applied effectively in 
many types of organizations, and new applications are being explored or adopted 
every day. The clearinghouses in our survey do no^ seem to be in the vanguard in 
this utilization and exploration, although there is some use of word processing 
equipment, particularly by the contractors, and some automation in database 
development and distribution. 

In discussions with selected clearinghouse personnel<-both Federal personnel 
and contractors--one could ^ense reluctance to use automation and skepticism about 
the relevance of advanced technologies to clearinghouse missions. One can readily 
understand some^f this reluctance. Computer and communicatiorls technologies are 
changing so rapidly that it is not easy for any information service organizatioft in 
either the public or private sector t**^ keep current witK the technology and be able 
to evaluate its relevance and potential importance to clearinghouse operations. 
Nevertheless'there is a very strong possibility that clearinghouses as a group may be 
failing to capitalize on automation to help develop more effective and efficient 
operations and thereby contribute to the achievement of their missions. 



" Clearinghouses operate, within the larger information environment in our 

society«*an environment in which a number of important trends are evident. For 

example, a number of information service organizations, includiag clearinghouses, 

were' created to help reduce perceived bottlenecks in identifying 'relevant 

information oq giyap topics. The'advent of online computer-based .systems, through 

which more than 400 producers of bibliographic, referral, numeric, and full-text 

databases, are distributing their products, lias largely removed this bottleneck so 

successfully, in fact, that, with- respect, to the availability of the professional 

literature, the c'hief bottleneck how is the lack of efficient means of obtaining 

'full- text copies of documents that have beeh " identified through- the online 

f 

bibliographic systems. * " ^ . 

Ef forts.are underway to remove this "new" bottleqeck. The number of services 
designed to^ facilitate document delivery is growing, and both secondary and primary 
document .fulfillment services are using automation to a much greater extent than 
ever before. The longei^term solution to the document fulfillment problem will 
occur primarily ..through the use of online and other electronic distribution media. 
The pioneers in this area have been the online services in the legal'and news-related 
fields, but some of the primary publishers are following their e^cample. ^Several 
experiments are* under way to test the technical and economic feasibility of 
providing the full text of scientific and technical journal articles online. ^ 

Not all pieces of the total technology needed to support full-text retrieval and 
document de^livery, particularly documents on demand, are economically viable for 
widespread u^, but by some estimates the technology will be able to support such 
use within the next five to ten years. The reshaping of the traditional roles of 
^primary, secondary, and other Information delivery organizations has yet to be fully 
envisioned, but it is clear, that these organizations will have to change and adapt. 
Economically and ottj^rwise suitable alternatives or complements to print-^on-paper 
will continue to be sought, because the rate of inflation in publishing printed 
materials continues lo outdistance the general rate of inflation. (Lancastei; 1980:4). 

Paralleling these developments .are equally important developments in 
distribution media that use broadcast and cable technologies to make the television 
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set a display device* For those clearinghouses whose mission is to reach the general 
public, the new "teletext** and "videotex" tpcbhologieq?^ offer the promise- of 
providing a great many new channels for outreach and dissemination into the home* 

It is too sooO' to forecast the role of the government, and clearinghouses 
specifically, as users these technologies, ^n awareness of these advances and' of 
the issues that surrounfl^ them, however, is mandatory in planning for^ new 
clearinghouses and the continual planning.^ that is part of managing existing 
clearinghouses* For examfile, some, observers would argue that these technologies, 
instead of having a democratizing influence and being able to broaden access and 
^ailability, are creating even more clearly distinguishable information-elite and 
information^poor segments of society. This may occur if clearinghouses follow the 
pattern adopted by other information services of charging for information made 
available via these ^ technblogies. . All publicly supported information services, 
includmg clearmghouses, need to consider both the merits and implications of this 
argument. 

Marketing^ Promotion, and Selling ^ 

Marketing professionals make^a distinction between "marketing" and "selling." 
They consider marketing to include activities associated with the definition of the 
target audienceT^finition of the product/service line, pricing, establishing sales 
performance targets, gathering market intelligence, ancJ performance evaluation. 
"Selling** is seen to encompass those activities that establish contact with^'the 
prospective user and persuade that user to buy (or use) the product or service. 

Most clearinghouses directors do not make these types of technical 
distinctions. To the extent that they even Use the terms cparketing and/or selling, 
they tend to use them as meaning approximately the. same thing, or they use the 
term **promotion," which has a less commercial connotation than "selling," and more 

One- an3 two-way transmission systems that use the television set as'" the 
"terminal." . o 



accurately representt those activities that are aimed at' informing their target 
audiences about the clearinghouse and its products^ or services* In this, report, we 
have treated some of the traditional marketing considerations (e.g.| defining the 
target audience) under the general rubrib of planning. Some of the other marketing 
consideratiqi^Sf as well as promotion and sales considerations, at J discussed as part 
of narrower •topics, such as publications pcograms and evaluation of clearinghouse 
performance. ^ . 

As in a number of other functional areas, marketing is presently perceived as 
being Vlosely related to many other clearinghouse ^operatiohs, particularly user 
services^ However, for planning purposes, it is useful to distinguish between those 
outreach activities that^ support other operational areas and objectives^ (e.g., 
contacts^ made to' Elicit papers for acquisitions) and those that^^re aimed 
'spectrically at informing target and other populations of the existence of a 
clearinghouse and of its products and services. 

A wide range of m*arketing techniques' are used by clearinghouse personnel, 
includiing: 

• exhibits at meetings and conferences likely to be attended by 
members of the intermediary' or ultimate target audience^ 

• presentations about the clearinghouse given at conferences; 
"direct mailings of specially prepared promotional material^ 

• news releases; 

• advertising in professional^ trade journals; 

• mass media advertising and 
. • exhibits in shopping malls. 

Some clearinghouses use a variety of marketing techniques, while others use a fairly, 
narrow set. 

The selection of -appropriate promotion and sales approaches and media will 
necessarily be guided by the characteristics of the target audienci^s. What is 
appropriate for reaching particular subsets of professionals, even/ in the same 
general field, may be different; and none of these approaches may be -ideal for 
reaching the general population. It may, in fact, not be worthwhile for. a 
clearinghouse to attempt to aim its promotional activities directly at the general 
public or to selected special pispulation groups because they are so difficult or 
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expensive to 'reach. Clearinghouse marketing resources may be used more 
efficiently for the distribution of promotional materials to intermediaries 
(organizations and individuals) with whom members of the ultimate target audience 
are likely to come in contact. ^ 

Relating marketing and promotional strategies; to the definitions. of target 
audiences and differentiating^ those strategies from others designed tc achieve a 
different set of information dissemination objectives involve complex problems* 
Until recently both marketing and sales activities have been neglected or, at best, 
carried out in a half-hearted way, sometimes' disguised in euphemjsms. This has 
been true for publicly supported institutions, and even within the private 
not«for*profit sectors* One reason for this situation is that some informction^ 
service managers and information service sponsors view selling as ^'unseemly". 
However, economic realities are helping to develop the counterbalancing attitude 
that the ability of an organization to alchieve its purposes requires that it also 
achieve some visibility. In this context, all marketing-related activities have 
become legitimate, although the resources being applied to -them are* still very 
limited^ by business standards. Only about 3 percent of the budgets of the 
clearinghouses in this study was being spent on marketing and sales activities. 
Whethier th|s investment level is sufficient to achieve a desired level of visibility is ^ 
an important^ issue that clearinghouse sponsors and managers must address. 

Planning and Evaluation 

Planning and evaluation are^ usually regarded as two of the most important 
fuM!:tions of management. While this entire report is an exposition on planning as it 
relates to clearinghouses, we mention the topic of planning here as a reminder of its 
role as a management function, .a function that Jtself must be planned. Planning 
may be reflected'in the specifications "contained in a contract work statement, in 
the recommendations from advisory groups, in the product of some other periodic 
review and evaluation process, and in olher processes and products. Evaluation is 
addressed in Chapter 6. ^ 

Operations and Operational Considerations 

The major sections covered in this chapter focus on increasingly detailed levels 
of planning. Li discussirtg this final set of considerations, the focus is on the major 



operational choices that must be studied. Some of the choices will influe:ice and be 
influenced by decisions made in some oi jhe ,earlier areas; others are primarily a 
matter of specifying operational procedures and guidelines* Ideally, the decisions 
that result from this and related^ se^ts of operational considerations will be 
documented by the clearinghouse invplvedy with appropriate sets ,of work^flow 
schematics. ^ 

This section covers three major and^ traditional clearinghouse activities: 
{ collection development and dissemination objective^ processing of materials into a 
formal collection; and qser services* A fourth major activity, publications, is 
covered separately in Chapter 3. 

Collection Development and Dissemination Objectives 

At the outset of this study it was necessary to define the concept of a 
clearinghouse in terms of a number of choracteristic activities that, collectively, 
distinguish clearinghouses from other types of information service organizations* 
Two activities that are of particulwr relevance to collection development and 
dissemination objectives are: 

t Acquiring literature* Must actively engage in the' acquisition 
of literature*bdsed information related to its focus or 
maintain a database representing records of literature-based 
information resources* Note: "literature" is broadly defined 
to . include audiovisual^ materials, speeches, description^ of 
organizations or programs, etc* 



• Developing an organized collection with appropriate access 
* ' tools* Must process and organize the acquired information 

into a collection with indexes and other tools to provide for 
systematic; searching' and access* 

The presenqe^of each of these activities was an irpportant criterion in distinguishing 
the targets of this ^tudy from information and referral services, "hot line" help 
units, and other information service entities that might also claim the name 
"clearinghouse*" 

In the planning process, the relative importance of these' two functions should 
be related to ythe overall goal of the clearinghouse, and a set of more specific 
objectives should be associated with the functions, to help determine what types of 
materials will be acquired by the clearinghouse and how they will be used* One 
objective may be to establish 2 small core collection of materials that will be 

t m 



supplemented, as needed, by materials in other collections. The collection and 
supplementary materials could be used as a resource by clearinghouse and agency 
staff in meeting user services and other information analysis and dissemination 
objectives. A different objective may bre to develop a comprehensive collection of 
materials and to disseminate widely information about the availability of these, 
material's through an online database and/or printed announcement journals. 

Collection Use * If the source materials in a collection are a key ingredient in 
the information dissemination objectives of the clearinghouse— as in the second 
example given above^-^then a number of dissemination vehicles can be developed to 
help ensure awareness and use of these materials. The vehicles include: 

• periodically issued accessions listings that provide citations or 
citations with abstracts to /tewly acquired items of general 
interest. These listings may be distributed as individual pieces 
or, alternatively, y/ithin newsletters, columns in journals, or 
other multi-purpose news-reiated item^ 

• periodically issued bibliographies (citations only or citations ^ 
^with abstrccts) on particularly relevant ("hot") topics that are 

distributed as individual mailing pieces or, alternatively, as 
part of other clearinghouse-released publication^ 

• regularly issued secondary journals containing citations or 
citations with ^abstracts and appropriate indexes of newly 
accessioned itenris that have been received and processed 
during some recent period of time. Cumulated indexes that 
ai'e issued on some regular basis (e.g., semiannually or 
annually, are also essential); and 

• citations or citatipns with abstracts that correspond in part or 
in whole with the items included in the alternative listed 
above, ^and are represented in computer-readable form for use 
in online retrieval. 

'Within each of these choices, a second level of related planning questions must 
be addressed. For example, the initial stimulus for preparing citations and abstracts 
in computer-readable form may be one of gaining efficiencies By creating a "master 
^database'* for a pliotocomposition-based publication of the desired journal and 
indexes. However, once the master database ia, in computer-readable form, it can 
be used as an online database to support many other management and service 
objectives, including ongoing analysis and monitoring of a collection (e.g., to learn 
the ratios of items on various subjects that are being acquired, as a . aid in 
selection, or to provide quality control checks for indexing consistency). The master 
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database can also.be used to provide online retrieval services for the staffs needs« 
A logical extension of the use of a database for internal retrieval purposes is to 
make it available for direct online access by the public, a ^tep that. has been taken 
(or is planned) by about 40 percent of the , clearinghouses in our sluay. If the 
database is likely to have some utility to those outside the clearinghouse, it can be 
made available online through onl or more online service organizations in the;public 
sector (d«g«, the National Library of MeJicine; the Chemical Information System of 
the National Institutes of Health and thelEnvironmantal Protection Agency) or the 
private sector (e.g., BRS, Inc^ DIALCX3 Information Service?, Inc^ or SDC Search 
Service). 

As indicated earlier in the discussion of "first cop/* costs, a large proportion^of 

a clearinghouse's operating costs can lie in the preparation of the collection. Thus 

having a "master database" from which multiple products are generated and multiple 

uses are supported helps to maximize the return on that basic investment. 
• * * ♦ 

Types of Materials to Be Collected The information dissemination objectives 
associated with collection development are established through interpretation of the 
answers provided to questions raised earlier-la ^his chapter. However, they will^so 
be answered in the context of the types of materials that are the focus of the 
collection development effort. The types of materials likely to be considered for 
inclusion (unless a particular type of information is a given starting point for the 
clearinghouse planning process) are given in Exhibit 2J. ' 

A clearinghouse that is established to deal prirparily with government«sponsored 
research in a particular area will weigh a different 'set of choices^ than a 
^ clearinghouse that intends to deal with statistical data or the full range of 
professional literature, regardless of source or format. For a clearinghouse focusing 
on the collection of government*sponsored research reports or other types of 
government*sponsored or government*collected information, the use of NTIS and/or 
ERIC for processing and dissemination must be Considered. If the journal literature 
is the primary focus of the clearinghouse collection, as it is for most of the 
clearinghouses in our sample, already existing outlets for processing, announcement, 
and distribution exist in both NLIyt and ERIC, as well as many private-sector 
services. 
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EXyiBIT2.3s ILLUSTRATIVE LISTS OF TYPES OF MATERIALS TO BE 
CONSIDERED. FOR INCLUSION IN CLEARINGHOUSE 
COLLECTION* 



Audiovisual materials (e.g., films; cassettes) 
Bibliographies 
Books 

Computer*readable tapes 
Conference proceedings and speeches 
Curriculum guides 
Directories 
Dissertations 
Journals 
Journal articles 
Legal documents (Federal) 
Legal documents (State) 
Manuals and handbooks 
Newsletters 
News clippings 
Pamphlets 

Program descriptions (Federal or Federally supported) 
Progra:Ti descriptiona^other) 
Reference tools 

Resegrch reports (Federally sponsored) 
ResearcFT reports (other) 
Referral sources (individuals and organizations) 
Statistical data (collected by the government) 
Statistical data (collected by others) 



♦ The scope of a clearinghouse collection also needs to be defined to include the 
matter of whether it will deal only with U.S.-related materials. 
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A relevant model for a clearinghouse that "feeds" materials into NTIS and NLM 
exists in the National Health Plannning Information Center* This model was 
appsrsntiy rejected in the planning for the National Clearinghouse for Bilingual 
^Education and the Nations! Clearinghouse on Aging, although the reasons for that 
rejection are' not clean The issues of overlap and duplication are discussed in more 
detail in Chapter 4^ but v^b should laote that the starting point for consciously 
accepting or rejecting overlap and duiPication is in the needs assessment and 
planning process. 

Economies of scale way be achieved in using capabilities already developed 
within the government to process, announce, and distribute materials* If these 
capabilities were um4 by other clearinghouses, could <and would) more resources and 
attention be given 'to information analysis and to tailored dissemination ef forta|? • 
The merits of the "tertiary database** concept, as developed by the now-defunot 
Women's Educational'Equity Communications Network (WEECN),* also needs to b&^^ 
studied further. The research should determine whether this is an economically 
satisfactory alternative for focusing attention on the literature in a very specific 
problem area while at the same time minimizing or eliminating the duplication of 
processing effort, across' public agencies and between clearinghouses and private 

'information organizations. 

< 

Processing of Materials in the Collection 

Levels of resources and effort associated with the information processing steps 
need to.he^qmmensurate with the intended uses. What is done to maintain a small 
resource library will be considerably different from what is done to support the 
_cJevelopment (?f a database that is to be made available.online dr used to generate a 
printed annobRCcment publication at regular intervals. In Exhibit 2.4, a nuniber of 
decision areas are described, to illustrate the range of choices to be considered in 
developing operational procedures. The illustrations of choices given in this Exhibit 
reflect the variety of practices currently adopted by existing clearinghouses. 
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* Through a series of agreements worked out with 13 public and private database 
producers, WEECN identified citations relevant to its scope through computerized 
searches of their databases and produced its own publication announcement using 
selected output of these searches (Butler and Brandhorst, 1980:174-176). 
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EXHIBIT 2,4: ILLUSTRATION OF PROCEDURAL DECISION AREAS 



Identification and Acquisitiona k 

Decide whether the desired materials must be defined through an active 
solicitation program to obtain typically unpublished or otherwise "fugitive" 
material^ If so, decide how potential authors and soOrces of. those meterials 
will be reached and with what frequency* 

Decide whether formal exchange agreements are needed with organizations 
that regularly or periodically produce relevant materials. 

Identify published sources, including online databases, that can be scanned and 
searched regularly for relevant items. 

Decide whether identified items will be purchased or only acquired on loan or 
through other means that do not require payment. 

Selection ^ 

Decide whether all items will be included in a single collection, or in one of^ 
several^ collections (e.g., a library, an informal secondary collection of 
materials maintained Mn vertical files for reference and referral use, and/ or a 

domputer*readable database). 

If 

Decide if any criteria, in addition to the relefvance of an'item to the defined^ 
scope of the clearinghouse, will be used in screening materials for inclusion in" 
the collection. If such criteria^ as quality, reproducibility, availability, format, 
methodology, and objectivity are to be used, formulate the definition of each in 
writing. . - 

Decide whether the clearinghouse collections will be linnited to U.S. or English 
language materials. 

•Decide whether the collection will cover only current- materials or whether 
older items of value will also be collected. If so, decide on the period of 
coverage and what exipeptions, if any, will be made* 

Preparation oiBiblioqraphio Records * . - 

Decide on the descriptive elements (e.g., titles, author) that are to be" included 
and tne standard .conventions or practices (e.g., Anglo-American Cataloging 
rules, MARC, ANSI) that are^to be adopted or adapted. 

.' : ^ 



EXHIBIT 2.4: ILLUSTRATION OF PROCEDURAL^ DECISION AREAS (Continued) 



Decide whether a classification system and/or subject terms will be assigned to 
each record. 

If subject terms are ^6 be assigned, decide whether these terms be drawn 
from a controlled vocabulary list or from the text of the title^ abstract, and/or 
dpcumento ^ . 

r 

If the access tooPbeing developed is a database, decide whether the facility of 
free»t,ext seardhing that will be provided by the online service (i.e., the ability 
to retrieve by words in titles and abstracts, as well a^ any other fields) in any 
way changes the requirement for developing a highly structured controlled 
vocabulary for supporting effective retrieval. 

If ^ controlled vocabulary is to be developed, decide whether an already 
existing vocabulary is applicable, as is, or whether one or more existing 
vocabulary tools could be adopted for use by the clearinghouse. 

Decide whether records will be typed (e.g., in the form of catalog cards) or a 
computerized database' will be^ created (e.g., through Optical Character 
Recognition or online data entry). 



2 



4G ■ 



A basic question that must be posed in considering the options available in 
preparing a "bibliographic" record (Ke*» a set of elements of information that 
describe and facilitate later location of a book, a journal articlci or a magnetic tape 
of statistical data) is whether automation should be u^ed* This question is usually 
posed in terms of: What is the "critical mass** or point, .at which it is worth 
automating? One of the reasons that no ^simple rule-of-thumb principle has been 
developed to ^^nswer that question is that studies contrasting costs of manual and 
automated systems are dealing with two essentially different outputs (e*g*i a card 
catalog versu^^'a^- computer-based catalog). It is l|kely, in fact, that for smaller 
coUectionsi it will b^ cheaper to develop and use a catalog or index manually* 
However, to answer [the ultimate question of whether that catalog or index is more 
cost-effective than - one created in computer-readable form requires that the 
capabilities for access, retrieval,^ and other manipulations (i.e., the effectiveness) of 
those two end products be compared. / 

Costa and some measure of value must be attributed to each of the'by-products 

A 

produced and uses served by the two different types of collection access tools. For 
example, a determination must be made of the costs for 'generating topical 
bibliographies manuall^as compared to generating 'them from an opline search. The 
value to management of being able to monitor and manage collection growth, 
analyze acquisiton trends, and conduct periodic checks on tHe quality of indexing 
must be quantified. Whether these activities Could be done readily through use of 
the manual tool is ah important planning consideration. Another factor to consider 
is the costs to user services for conducting a literature search manually, as opposed 
to online. Research findings suggest that use of an online database is more 
cost-effective than a manual approach (Wanger et al., 1976; East, 1*980). However, 
if no particular value is attrib,uted to increased facility in the, use of a collection 
tool, automation will always appear to be the more costly alternative. 

N 

User Services 

A taxbhomy of types of user services associated with clearinghouses was 
developed iPor purposes of collecting and analyzing data for our' survey. A further 
refinement of that taxonomy 1^ presented in Exhibit 2.5. 

Reference and referral services may be characterized in terms of the degree of 
responsiveness, and in terms of how ^ctive the clearinghouse will be in seeking out 
opportunities to provide these services. A clearinghouse that responds to all 
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EXHIBIT 2.5: ILLUSTRATION OF MAJOR TYPES OF USER SERVICES 

\ 

\ 



REFERENCE^AND REFERRAL SERVICES 

Document delivery \ ^ 

D6cument loan • * 

Responses to requests «f or: 

information about the clearinghouse - " 

information about an agency program 

information on "what is available ' on this topiCy" for which 
' , preaSv'embled lists of references and/or* copies of materials have 

been prepared 

quicks-reference questions that require* lers than 30 minutes to 
obtain an answer through searching a reference tool| contacting one 
or a few other organizationsi or obtaining list 6i a few references 
(to the literaturoi people^ programs, or organizations) 

research on a problem' that requires more thah 30 minutes to obtain 
an answer and requires preparation of some distillation/synthesis of 
findings from readir^gs in multiple sources or contacts that are made 
with other organizations 
^ * * 

^ computerized or rhanual literature searches, to generate a 
bibliography tailored to the .particular reguest, using Ihe 
clearinghouse database (or indexes) and/or databases (or indexes) 
produced by other agencies and organizations. 

CONSULTING/TECHNICAL ASSISTANCE . 

Sponsorship^ of conferences and workshops on topics or problem areas within the 
scope of the clearinghouse! for members of the target audience (e.g., to 
facilitate communication among those working in'T:he same area); ' 

Education and training seminars aimed at helping the target audience or 
intermediaries (including library and information services personnel) to use the 
clearinghouse! its publicationsi or its databse. 

Consulting and technical assistance to help an individual or organizationi or 
groups of individuals and orjganizationSi to apply information to particular 
situations and problems* 
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inquiries with standardly pre-as^embled lists of references will be perceived as less 
responsive than a clearinghouse that handles each request individually and tailors 
the response appropriately* Howeveri if reference and referral services are not 
perceived tio b^e the most important instrument in meeting clearinghouse 
dissemination objectives, it may be entirely justifiable for the* clearinghouse to 
provide only minimal levels of such services. On the. other hand, referral services 
are almost mandatory in a olearinghouse operation^ Requesters are not likely to 
have an understanding of clearinighouse objectives, operations, and limits, and they 
are likely to^expect a government-sponsored agency to extend itself, at least to the 
point of referring the requester to a more appropriate information source. 

. User services need- not be' thought of simply in terms of passivity and 
responsiveness* Many opportunities fur outreac^exist in this area as a number of 
clearinghouses seize on these opportunities to provide a very targeted service. For 
example, sojne clearinghouses volunteer their services to other organizations that 
operate in the field (e.g., agencies, foundations, professional associations). They 
may support the research and publications dissemination efforts of these 
organizations or may develop materials for use in their conferences, meetings, and 
special events. These special materials (e.g., bibliographies or summaries of major 
research and development activities) are tailored for greater effect to the theme or 
special focus^of those meetings. 

' For each of the reference and referral services that will be provided by the 
clearinghouse, it will be necessary to define relevant policies and procedures. For 
'example, policies will be neede.d on telephone follow-up to mail inquiries (to help 
ensure that a request is understood) and on adherence to copyright restrictions 
related to the copying of Imaterials developed by athers. Similarly, procedures must 
be defined for receiving a\id screening requests (e.g., ty mail or phone) and. routing 
them to appropriate staff, ^and for the' use of stan^^ard letters and forms (e.g., for 
maintaining records and for inclusion in the response package). 

Irr addition, consideration ^eeds to be given to methods of supervision and load. 

s 

balancing. Regular staf^meetings are a means by which requests are discussed in a 
groyp, ideas for fulfilling those requests are shared, and targets are set for 
allocating time. A clearinghouse can strive for quality assurance through regular 
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review of responses prepared for users and through use of. evaluation/ feedback forms 
sent to usersMri response pac><ages. 

Another ^area of user services involves giving technical assistance to individuals 
and organizations in their use of information for specific purposes* This service is 
not a traditional function in most clearinghouses surveyed. Strategies and models 
for such, linkages exist (e.g., in the NIE educational diffusion programs), but 
determining their applicability to each field and problem area covered by a 
clearinghouse is beyond the scope of this study* . 



\ 
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COST-RECOVERY ISSUES AND CONSIDERATIONS 

• * / 

Introduction 

The feasibility of operating ^overnment*supported informatipn services on a 
co8t*recovery b^sis has^^been discussed for many years by proponents and opponents 
of this approach^ but no final decision has been made about implementing it on a 
government-wide basis* In an era of limited Federal funding, cost recovery is again 
being suggested as a means for continuing such information seYviceSi but at a 
reduced cost to the government. This chapter presents information to inform 
policynrtakers by highlighting the following topics: 

• the demand for cost-recovery practices ; 

• economic theories pf information ; 

• current practices among information services ; and « 

• ' considerations in .implementing a cost-recovery policy. ^ 

The Demand for Cost-Recovery Practices * 
» *> " ' 

The Federal governhnent has long had a policy of charging recipients of 

government services If they receive special benefits above and beyond those 

accruing to the general public. The policy is specified in Title 31 Section 483A 686A 

of the O.S. Code and in.4he OffFce of Management and Budget (0MB) Circular A-25, 

issued in September 1959. The circular states that a charge should be made to each 

identifiable recipient of a" government service from which a special benefit is 

derived; to recover the full cost to the government of rsndering the service. * 

In the past few years, pressure has grown to apply this policy to information 
services. A 1979 report by the General Accounting Office (GAO) recommended that 
the Director of 0MB direct attention to the development of a clear cost-recovery 
policy and to» assuring its consistent implementation (GAO, 1979:33-34). In June of 



1980, 0MB issued a draft circular on the improved management and dissemination of 
Federal information* It specified that information made available through 
mechanisms other than the dgpoettory library system be provided at a price. that 
recovers all costs to'^ the government associated with disseminating that 
information. Although the^circular has not yet been published in final form, 0MB 
officials indicate that they expect a requirement of this type to be issued. 

Cost recovery for government^supported information services has become 
increasingly relevant in recent months* As reductions in proposed Federal spending 
for human services programs are discussed, it appears likely that some of them will 
affcgt agency spending for information services in those areas. In addition, OMB 
and GAO have activated studies of government information services in response to 
the Paperwork Reduction Act of 1980^ Officials at both agencies indicate that cost 
recovery is one of the topics to be addressed. They are aware of the limited extent 
to which government-supp9rted information services currently recover costs and 
indicate that previous levels of Federal financial support for information services 
may be inconsistent with current priorities. Two key considerations in the 
discussion of cost^recovery practices for information services have to do with 
whether such services constitute a "public goocf' and, if accost-recovery principle is 
to be applied, what the basis for fees should be. These are discussed in the next 
section, in the context of economic theories and practices. 

Economic Theories and Practices Related to Information 

A discussion of the feasibility of cost recovery for information services should 
be grounded in an understanding of ^ the information market and of alternative 
pricVq strategies. The intent of this section is to provide this background. 

The Information Market 

A market economy in which goods are sold and bought is thought to work well in 
circumstances where people pursuing their own self-interest act in a way that is 
consistent with the wants and needs of others. For example, it is in the interest of a 
manufacturer to produce goods as efficiently as possible and of a buyer to purchase 
those goods at the lowest possible price. Thus, the interests of buyers and sellers * 
are mutually reinforcing, §nd the decisions of buyers and sellers determine the 
amount t}f goods and<services sold and the prices at which they are sold. 




There are some goods and services that are provided to the public through 
mechanisob ot>ier than the marketplace. The best example is national defense, 
which -is something that is provided to the entire populace and should therefore be 
paid for with tax revenues. When goods or-^^services are provided outside the 
marketplace, they are khown as "public" goods. 

Economists define several conditions under wf^ieh a good may be considered a 
public good. Some of the conditions have already been'innplied above: that t*he good 
is provided to the entire populace and that those who do not pay for the good cannot 
be exclUdedvfrom benefiting from its existence. For example, when fire and police 
protection are\available to a community, all citizens recey/e such prot;jBctioh. 
Another condition under which goods may become public goods is^if private R|operJy 
rights are difficult to protect. " "/ ^ 

During the past several years there has been intense debate on whether 
♦ 

^formation is a "national resource" that should be. treated as\a' public-good or a 
commodity that should be treated like other comrnodities that are b'bijght and sold in 
the marketplace. There are some special characteristics of information that 
underlie the debate: \ 

• information is an intangible that' can be made available in 
mairy media; 

• information is hot consumed by use; it can be resold or given 
away 'With no diminution of its content or, in most c.ases, its 
valu^ 

• the price of information bears little relationship to the costs 
of making copies available. The "first copy*; cost is likely to 
be the major cost, with reproduction costs being relatively 
minor; and 

• information has Value in the marketplace, as a capital 
resource, an essential tooi fcr decisionmaking, and a means for 
better management of tangible resources (Olson, 1973:8-12). 

Although information services have some of the characteristics of a public 
good, they are also succeeding in the^/narketplace as commodities. Particularly in 
technical and specialized, fields, there is a growing willingness to pay fnr 
information* These markets are served by information enterprises that sell 
newsletters, indexing and abstracting publications, current awareness s'ervices, and 
• other information services, often at a substantial price. For example, -many 

a 

organizations spend over $10,000 a month for corf^mercial online reference services. 



The users? willmgnesd to pay is interpreted as a reflection of the growing recognition 
of tfieCxvdlue of specialized information* The development of viable commercial 
information services is seen as an indication that at least some information can be 
treated as a commodity* 

Costs and Pricing of Information Products and Services 

Cost Categories* There are various categories of costs^that are incurred by 
. information services* These include: 

information generating costs incurred in collecting or 
^ organizing the information input; 

design and development costs associated with producing a 
publication or providing a service; 

fixed costs involved in maintaining the operation^ including the 
costs of equipment, utilities, and space; 

reproduction costs associated with printing or photocopying 
reports, search results, or other written material^ and 

distribution costs for marketing, handling, and mailing* 

Average cost pricing establishes a price based on recovering the fixed costs, the 
reproduction costs, and the distribution costs* Marginal cost pricing, aimed at 
recovering the costs associated with^.each additional unit of service, recovers only 
the distribution and reproduction costs* Both of these pricing strategies assume that 
the true cost of the publication or service is known* When '•ecords are not detailed 
enough to support such price calculations, other pricing strategies nnay be used* 

- \ 

Calculating Prices* Two methods used when all costs are not known are 

formula pricing^and target pricing* In formula pricing, the manufacturing (e*g*, 

printing) cost is multiplied by k previo^l^ determined factor; publishers sometimes 

use a formuia of five or six times the manufacturing cost to 'price books. Target 

pricing, a method used by utilities, involves determining in advance a price that will 

yield a desired rate of return on the total cost at an estimated volume of demand 

(Zais, nd: 146-47)*. 

'* » 

Price discrimination is another pricing strategy used by some information 

.services* This practice involves charging different ^ees to various categories of 
« 

users. Price discrinnination is considered feasible only when the market is 
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segmentable and recipients of lower priced services are not likely to resell the 
information to those groups who are being charged more. 

For government agencies, seifectlon of the priring mechanism may be a political 

or philosophical ' decision: A major factor in government agencies' pricing 

calculations is the objective to be met by distributing the information. Many 

agencies regard the cost of generating the information (e.g., support of research and 

information-gathering programs and dissemination of findings) as a necessary 

program expense and consider information production the reason for their 

existence. They argue that unless policymakers, researchers, and administrators are 

informed about *prior work, there will be unintentional duplication and/or pursuit of 

fruitless avenues of investigation. Therefore, supporting the development of an 

appropriate information^ base and the means for storing, retrieving, and 

disseminating the information are valid and necessary government activities. This 

philosophy leads to the view that, if costs are to be recovered, only the marginal 

costs incurred in disseminating the information should be recovered. Another 

agency might maintain that the purpose of its program is to inform the public about 

a social problem or societal priority, such as the dangers of high blood pressure or 
# 

alcohol abuse, and that its mission makes it essential for the government to absorb 

•> 

aU costs, including those related to dissemination. 

Another point of view suggests that secondary literature (i.e., abstracting and 

indexing publications, bibliographies, and related reference materials) should be 

priced low because they have limited informational value, except as a means for 

directing users to the primary literature. They contribute to increasing the demand 

for primary literature, thereby decreasing its average cost. This view holds that 

other considerations are involved in pricing tertiarv literature (e.g., synthesss, 

f * 

state-of-the-art reviews, and similar summaries) because this literature is expensive 
to produce and does not necessarily stimulate the demand for primary literature 
(King Research, Inc., 1980:37). ,„ - 

Types of Fees. Several mechanisms are used for collecting fees for information 
products and services. Among the more common methods are: 

• a fee-for-service, based on the level of effort required to 
produce the information, with the fee paid after service is 
provided (e.g., a search charge based on computer connect 
time and number of "hits" or a fixed price for .a specific 
publication); 




r a flat rate established in advance for a specific type of service 
and ^paid for) befdre or after receipt 'of the service (e.g., a 
charge of $x,per search); and 

9 a subscription fee, paid in advance, to establish eligibility or 
to cover ufl to a certain level of service (e\g., an annual charge 
of $x, which entitles a subscriber to a cerljain number of free 
or reduced^cost searches during the year/7 

An' important factor that providers of information products and services 
consider in selecting the fee collection method is the transaction cost. Transaction 
costs are those costs ir^curred by the seller in ^billing and accounting for the fees 
collected and the costs incurred by the purchasers in ordering and paying for the 
item. F or inexpensive items, the transaction cost may be as high as, or higher than, 
the p^duction cost« Because transaction costs can be substantial, sellers tend to 
prefer the least burdensome payment method. For this reason, subscription and 
prepayment plans offer some advantages. 

Current Cost-Recovery Practices 

Many directors of government-supported information services maintain that 
full-cost recovery is not feasible for their operations. The arguments that they use 
to justify their position are .summarized in Exhibit 3.1. The views expressed by 
proponents of cost recov3ry are also summarized in the Exhibit. The literature and 
the interviews conducted for this study present both points of view, but no definitive 
research has been conducted to verify the potential impacts presented on both sides 
of the issue. It is, therefore, instructive to describe current practices among 
government-supported and private information services. 

As part of this study, we gathered information from 27 clearinghouses about 
their cost-recovery practices. The three clearinghouses in the sample that are not 
government-supported rely on user fees to recover all costs. Of the 24 
clearinghouses in the sample that are totally or partially supported by public funds, 
14 recover some of their expenses from fees, and one operates on a 
full-cost-recovery basis. The latter is the National Technical Information Service, 
which is legislatively mandated to be self-supporting* It differs from the other 
organizations in the sample, because it reproduces reports developed by other 
agencies and does jrot generate synthesis and analysis publications. Nine 
clearinghouses, primarily those in the health field, recover no costs through user 
fees. 
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EXHIBIT 3.1:- ADVANTAGES AND DISADVANTAGES OF COST RECOVERY FOR 
PUBLICATIONS AND SERVICES 



Advantages of Cost Recovery 



Disadvantages of Cost Recovery 



Shifts the costs from the 
sponsoring agency (and 
the general public) to fhe 
specific users of the 
information 

Reduces or eliminates some 
Federal spending 

Permits more people to be 
served by increasing the 
financial resources of'the * 
clearinghouse 

Produces more accurate targeting-, 
of information, because only 
those who are interested will 
request it 



Re(|uces the distribution of 
unnecessary or unwanted 
information, because poorly 
received services and publica- 
tions will be eliminated > 

Provides an incentive for greater 
responsiveness to user needs and 
better selection of topics for 
publications 



Costs cr^ shifted, not 
eliminated, when user^ are 
government agencies, and costs* 
may actually increase, because 
of the addition of processing 
costs for handling transactions 



Reduces demand, because of user 
inability or unwillingness 
to pay fees 



Leads to less adequate dissemi- 
nation of information that has 
been designated a government 
priority area but that may not 
have a significant market 



Reduces willingness to share 
and exchange information, at no 
charge, when suppliers of 
information cannot get anything in 
return 



Improves the environment for 
commercial enterprise, because 
all services can compete 
equally, if government subsidies 
tor some services are 
eliminated 



Increases time-lag in proces- 
sing of requests to handle 
billing 

Creates potential billing and 
bookkeeping problems, as well 
as difficulties in recovering 
fees from delinquent accounts 

Unless money reverts to the 
clearinghouse budget, provides 
no incentive for improving 
products and services 
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These findings are consistent with those produced by other studies* Of 111 
government«*supported scientific and technical information services studied in 1980| 
57 percent charge for at least some publications or services* Charges are most 
common for monographs, non-print rnediai and computerized searches. Newsletters) 
advertising brochuresi and referral to other sources are typically prjvided free of 
charge (King Research, Inc., 1980:40-48). A study of 17 subsidized centers for 
selective dissemination of information reports that a mean of 30 percent and a 
median of 14 percent of all revenues are derived from subscription payments (Zais, 
nd:174-177). A GAO report indicates that information centers typically':recover 
about >15 percent of the costs associated with use!* services (GAO, 1979:23). ^ 

These ' data indicate that although some costs are recovered, 
government-subsidized information services tend to rely heavily on their subsidies. 
Many clearinghouse directors express doubt that full-cost recovery is feasible for 
their operations because of the inability of users to pay for services, difficulties in 
establishing and maintaining billing operations, and the highly targeted audiences for 
some of their services and publications. They maintain, that marginal costs for 

\ • synthesis and analysis publications and search services can. be recovered, but that 

the development costs for these publications and for establishing databases cannot 

•'l be recovered. For question-answering and referral services, where there is an 

immediate need for a response and the answer is provided without sub^l:antial time 
devoted to research, directors believe that it wbuld be impractical to charge a fee. 
They assert that in these cases the transaction costs would exceed the value\p^the 
revenue taken in. The same viewpoint is expressed concerning the distribution of 
pamphlets, brochures, and short bibliographies that serve both public relations and 
information functions. ' > ' ^ 

A survey conducted by King * Research, Inc. as|<ed respondents at 
government-supported information organizations that charge for their publications 
and services what effect user charges have had on the organization. Of 100 
respondents, 5 estimated that there was some increase in the number of users, 31 
stated that there was no change in the number of users, 49 estimated there was a 
decrease in the number of users, and 13 did not know the effect. A similar 
distribution was shown with respect to the impact on the number of requests 
received. Although some respondents indicated that the quality , or size of 
publications had increased as a result of the imposition of user fees, most reported 
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that no change had occurred and that no major change in user satisfaction had been 
noted. A majority, responded that bookkeeping and accounting costs had increased 
because of the institution, of user fees (King Research, Inc., March 1980: 52-54). 

^ipsonnel at selected private information services and representatives of the 
information industry were contacted to learn about their cost-recovery practices. 
One database producer received government grants to partially subsidize start<-up 
costs for the online database, but the grants covered only a portion of the actuisd 
cost. Other services contacted indicated that they had received no government 
subsidy. Typically, the commercial sector relies on investment "Capital or* pbofits 
from other product lines to offset losses until a new service becomes 
self-supporting. To provide the potential for. initiating new services, private 
information 'services often charge substantial user fees for established services. 
Where there is no competitive service,* the fee may be based on what the market 
will bear, in other words, on the perceived, value of the service to the user, rather 
than on actual costs for providing the service. 

To illustrate the cost differential betw^een government-sponsored and privately 
supported information services, we explored charges for leasing databases. Tapes of 
the database typically contain literature citations, or citations and abstracts, and 
are updated regularly. Several government-supported clearinghouses, including 
ERIC and the Clearinghouse on (bhild Abuse and Neglect Information, make copies 
of the tapes of their databases available to commercial vendors and other interested 
institutions at a nominal pi^ice to recover the cost of reproducing the tape. There 
are also no royalty fees assessed for use of the data online by users. 
Norvgovernment^ information services, by contrast, sell their tapes to interested 
institutions for a limited period of tinne and either apply a standard fee or negotiate 
that fee/with each lessee. The fees, among those services contacted, range from 
$1,000 to $6,000 per year for one year of the tape. Often there are additional 
charges for each use of the database and/or each "hit" (i.e., each citation selected 
for offline printing). These royalty fees are passed on to the user in the 
connect-hour prices and charges for offline (and, sometimes, online) printing. 

Many private services operate to a large extent on a pre-payment basis. They 
rely on annual subscriptions to index and abstract services and current awareness 
services and charges for databases as sources of revenue. Some services also have a 
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prepaid search service, in which the user pays *a fixed fee for a prescribed number of 
searches during the year* Deposit accountSi which involve^ drawing against funds in 
an established account as requests are received, represent another mechanism that 
has been u3ed. (A few government organizations, such as GPO and NTIS, also use 
the deposit account mechanism*} These arrangements reduce the need to bill for 
services as they are provided, and they simplify the entire transaction process. 
However,* they are appropriate only in those cases where users are repeat 
customers* For one-time users, some private organizations permit charging services 
on a credit card to prevent the time lag involved ifj waiting for payments to arrive 
before service can be rendered. 

Whether clearinghouses could follow the practices of private information 

services cannot be determined \ykhout knowing If the populations served by the two 

ft 

.groups differ in ways that would affect their ability or willinyness to pay. Some of 
the prix^te services believe that the population using their services is not 
substantially different from the population of clearinghouse users. For example, 
man4 academic and drug company researchers are* users of private scientific 
datalj^ase^ they are also users of the government-supported MEDLINE database. 
However, the extent to whith public employees, whose agency information budgets 
are limited, and the Itsy puolic could afford or would be willing to pay the fees 
charged by private services has not been thoroughly explored in any previous studies. 

In addition* to the ability and willingness of current users to pay- for 
clearinghouse services, procedural ^nd pollby considerations must b^ addressed 
before a cost-recovery policy can be fully implemented. These topics are presented 
in the, following section. 

Considerations in Implementing a Cost-Recovery Policy 
Policy Questions to be Resolved 

Three major policy questions that have not been resolved in previous OMB 
circulars, GAG reports, or other policy statements must be addressed before a 
cost-recovery policy can be implemented.' They are: 

• What publications or services should be sold on a cost-recovery 
basis? , * - 

• What kinds of costs should be recovered? 

• Who will receive the o^esulting revenues? 

Each of these questions is discussed below. 
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Publications or Services to Be Provided on a Cost-RecoveW BasiSt The concept 
of cost re.covery as described in^ existing law and 0MB Circular A^25 generally 
implies tHat costs -should be recovered in all cases in which the recipient of services 
derives a special benefit over and above that obtained by the public at large. 
Directors of government-supported clearinghouses assert- that applying this policy is 
impractical unless distinctiorj^ban be drawn about the categories of services to be 
covered by the policy. T^Jl^ cite several types of examples, such as the following, 
to support their view. 

• Some clearinghouse materials, including brochures, short fact 
sheets, and mini-bibliographies are intended to serve . as 
advertising' for the clearinghouse or as elements of a broad 
outreach program* Non-government information services 
provide similar types of mar.erials free oT charge, and it is 
unfeaUstic to expect potential users to pay for these items. 

• . If a charge were made for the types of materials identified 

above, the transaction costs for billing and accounting could 
exceed the revenues taken in, because processing a large 
volume of small, orders is very expensive, ' r 

• Some clearinghouses were established to support a government 
mission, such as enhancing the public's awareness of the 
dangers of a particuldiV disease* The public may place no^ 
priority on this information, but the government has decided it 
is an important socia] concern. Expecting the public to pay 
for , information that is basically a government priority is 
considered unrealistic. 

o 

These views were raised in discussions with personnel at 0MB andGAO to learn 
how they would apply cost-rec6very principles to the activities of 
government-supported information ^clearinghouses* Their responses indicate a need 
to make case-by-case decisions in applying a general cost-recovery policy. They 
agreed that previous policy enunciations, including the 1979 GAO report and the 
1980 draft 0MB circular, do not distinguish among publications and services, and 
they noted that it is difficult for a short policy statement to address all 'possible 
circumstances. 

The primary point er^pressed by these officials is that high-level policy-setting 
is needed regarding what government information services should be provided free 
of charge. They are concerned that high-level agepcy officials may be unaware of 
the amount and J:ype of information currently provided free of charge and that there 
is no consistency, either within or across agencies, with respect to fee policies. 
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They maintain that these decisions should be made by responsible agency officials 
and justified in agency budget submissions. In some cases^a cost-recovery objective 
may interfere with achieving other, more important. government objectives and may 
therefore be inappropriate* In ither cases, these officials assert, agencies may 
decide that enough information is available from other sources to eliminate the nei^d 
for additional government-supported information generation and distribution. 
However, if an agency determines that continued provision of information is 
desirable and that the costs cannot' be covered with agency funds, cost-recovery 
procedures should be implemented.' 

Categories of Costs to Be Recovered. As indicated earlier in thi? chapter, 
ttaere are costs associated with eact> stage of providing an information service, 
including the cost of collecting information, designing a publication,' reproducing it 
and distributing it. There are also fixed costs for maintaining the facility, 
regardless of the volume of - service provided. Clearinghouse^ personnel ere' 
concerned^ about which of these costs must be covered by user charges. Many of 
them assu|ne that fiiU-cost recovery implies recovery of all cost categories, a policy 
they believe may not be feaisible. . 

The original mandate for cost recovery, as stated in Title 31 of the U.S. Code, 
indicates that costs in all categories should be recovered. It states that all services 
performed for any person or organization not engaged in official government 
business should be 9elf-su3taining to the greatest extent possible and authorizes- 
establishing fees that take into consideration direct and .indirect costs, value to the 
recipient, and public policy or the interest served. A 1974 Supreme Court case, 
National Pable Television Association vs; the U.S., interpreted the law to mean chat 
the fee cpuld not exceed the actual value received by the user. Thus indirect costs 
for activities that benefit the public at large, rather than the indiViduai user, and* 
the costs incurred in establishing the entire program were not to be included in the , 
user fee. The xiase determined that the fee should recover only the costs of 
providing the specific service (GAO, 1979t?J3-2 1). 

This modified interpretation is reflected in the draft 0MB circular issued in 
1980. It states that user fees should cover the costs associated with dissemination 
of the information, including printing, process ing^ and retention costs, but excluding . 
costs Incurred in producing qjr creating the information (0MB, 1980:6). This view 
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was reiterated in discusisions wtt|i 0MB personnel responsible for drafting' the 
proposeg circular*. They indicate ^hat bosts incurred as part df normal program 
operations (e*g«9 for research or development) need not be recovered^ it is the 
incremental cost associated with dissenninatiorh-in other words, the marginal costs 
of the information service— that should be recovered* However, if a database or 

^information service is created specifically to make the information available to the 

. public at large, then recovery of all costs may be appropriate* 

Some literature pertaining to disseminating government-generated information 
supports the view that only marginal costs should be recovered (Kertesz, nd:3-4; 
Olson, "1973:12- 13; DeGennaro, 1975:366)* Some theorists suggest that, in calculating 
costs, only the additional expense of the specific service provided should be 
- considered. The input costs (i*e*, developmental costs for synthesizing or organizing 
the information) should not be included because they represent the primary 
information transfer functior^-- the raison d'etre of'the information center (Kertesz, 
ndi5-4). It should be noted that charging only the marginal cost fo^ information 
services differs from the practice common among private' information services of 
charging to repover all costs. However, to the extent that government-sponsored 
clearinghouses provide services that go beyond organizing the extant literature by, 
for example, synthesizing lihe literature in state-of-the-art reviews, they also incur 
costs that are not normally a part of the operating expenses of private information 
services* 

Another factor that could affect pricing policies of government-supported 
clearinghouses is that many of them are targeted to serve relatively small 
audiences* If a small group were asked to pay the average cost of information, as 
opposed to its marginal cost, the cost per unit would be higher than most 'jsers 
would be willing to pay* 

Agency to Receive the Revenue from User Fees* Several clearinghouse 
di^^ectors have observed that current laws provide a disincentive to collecting fees 
because, if an organization operates on appropriated funds, any revenues taken in 
must be returned to the U*S. Treasury. Under this arrangement, if cost recovery is 
practiced, the clearinghouse incurs additional billing and accounting costs but 
receives no benefits from the funds that are collected. Neither the 1979 GAO 
report nor the dr>aft 0MB circular addresses this problem, but officials of both 
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agencies are aware that this^is a significant disincentive* Although no attempt has 
been initiated to change the law, 'there is a willingness .to consider alternative 
approaches. One such alternative is to permit the program to recoup user fees up to 
the point where they "equal operating expenses; any revenues in excess of costs would 
revert to*^the U.S* Treasury,, ^ 

It |l30uld be noted that . some government organizations operate on a 
co.st*recovery basis bemuse they -do not receive appropriations for their normal 
opjprating expenses. For example, NTIS's^ authorizing legislation specifically 
requires;it,to operate on revenues derived from user fees, and GF^ operates under a 
provision that requires prices to Be based On the cost of printing, plus 50 percent to 
cover other operating^ expenses. TheCfe two organizations also retain the revenues 
they receivo from the sale of any clearinghouse publications they distribute. 

Because^ nrwst government agencies are unable to retain revenue' from user fees, 
several contractor-operated clearinghouses have contractual arrangements whereby 
ail user fees are retained by the contractor. In some cases, the fees are placed in a 
revolving fund to cover specific expenses, such as printing costs. In other cases, the 
government's reimbursement Jo the contractor is adjusted downward in an amount 
•eqiial to the retained revenues. Although these arrangements have proven 
satisfactory where they.have been tried, other government Project Officers express 
doubts about the effectiveness of this approach. They assert that if there were a 
move toward more complete cost recovery, the government would, lose control over 
the contra':tor because the contractor would have an independent source of funding* 
To the extent that the clearinghouse is intended to support a specific programmatic 
mission, this loss of control would be undesirable. They prefer an ai^rangement that 
would permit revenues to revert to the government program funding the 
clearinghouse^ ' - 

Implementation Problems to be Resolved 

In addition to the three policy considerations discussed above, several 
operational problems must be resolved before a more comprehensive cost-recovery 
program can be practicrable. These problems are reviewed in the paragraphs below. 

Nv *ed for Accurate Cost Data . Cost recovery depends on an accurate and 
complete accounting of the costs of the operation, by function. This information is 
needed to establish fees that reflect the cost of each type of publication or service. 
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This study revealed that'few among a sample of 27 clearinghouses maintain records 
in a- form that permits tracking costs by function. Some contractor-operated 
clearinghouses use a task-based cost accounting system that reflects contractually 
required tasks, , rather than information processing function^ other 
contractor-operated clearinghouses do have a cost accounting system that tracks 
costs in a manner thjat is relevant to establishing fee schedules. 

The problerfi is more severe in the case of clearinghouses operated enfirely or 
largely by goviarnment agencies! Only limited cost data are available for the eight 
clearinghouses of ^ this type in our sample. In three cases, the clearinghouse is not a 
separate line item in the budget, and staff are upable to provide any estimate of the 
organization's biffiget. Four can provide estimates of the total budget, and another 
can provide total cost* data for the program (broader than the clearinghouse) and for 
those clearinghouse functions that are contracted out. At the level of costs by 
function/onty incomplete data can be provided, -.Where a function is contracted out, 
th^ contract cost is knov^n. In some cases, clearinghouse personnel can estimate' 
direct labor charges to a function, such as staff time devoted to management or 
user services, but they do not have data on the indirect costs attributable to the 
• 'function (e.g.; telephone, supplies).- In other cases, some of the direct costs of a 
funption are' known, such as the costs for purchasing journals, but the labor costs 
associated with the activity are not known. 

Government-operated clearinghouses also do not have cost data for functions 
and services provided ^ the government, such as rent and utilities. One of the 
costs that has proven difficult for some clearinghouses to obtain is the cost of GPO 
printing. Several clearinghouse directors indicate that they do not regularly receive 
a report on their bills for GPO printing. In checking with GPO, .we learned that GPO 
works through each agency's Printing Officer or Chief of Publications. Printing 
arrangements and billing are handled in a centralized manner, and GPO does not 
usually report information to the organization that initiated the job. Each agency's 
printing or publications office is left to set its own policies on divulging cost 
information to the originating organization. Although it may be difficult for GPO to 
report costs back^ to- ail of the originating organizations, some anangements 
whereby each agency's printing. or publications office could provide a report back to 
.the originator of the document is essential to a cost-recovery program. 
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Need for Marketing to Build a Clientele , Commercial information services 
^realize that it often requires several .years for a new service to become 
self*supporting« In the interimi commercial enterprises rely on investment capital 
and/or on cost sharing, in which already profitable services subsidize- those that have 
not reached the self-supporting or profitable stage. If fuU-cost recovery were 
implemented for government-supported clearinghouses, it is not certain that they 
vyould initially have any publications or services that would generate sufficient, 
revenue in excess of costs to subsidize their less "profitable" enterprises. Limited 
demand may be a problem that is particularly acute because reliance on user fees 
requires extensive marketing to generate high demand and thereby permit reliance 
on user fees (Weisman, 1972:133). The data we. gathered in our interviews with 
clearinghouse personnej^ indicate that^e^j^'^f'the^overnment- funded clearinghouses 
h^e substantial expertise in either marketing or sales. Also, they have not had 
large budgets for these activities and, therefore, have been limited in their selling 
efforts. One complication to possible expansion of sales activities is that 
commercial and non-profit information services are likely to take a very dim view 
of aggressive selling by the government— particularly in competitive areas. 

Application of Cost Recovery When Intermediaries Are Involved . A final 
problem, and one that may prove particularly difficult to resolve, is how to 
implement cost-recovery procedures in cases where clearinghouses work through 
intermediary agencies, such as professional associations or state and local service 
providers, to reach their target audiences. This approach is used when the target 
audience is difficult to reach and/or widely dispersed (e.g., drug users, youth). The 
intermediaries provide a vehicle for distributing clearinghouse-produced information 
to such audiences. In that role, they provide a valuable service to the government, 
and a3king them to pay for the materials \nac they distribute may not be realistic. 
Also, having the intermediaries collect fees and then reimburse the clearinghouses 
may be, cumbersome, and they would have no incentive to perform this function. Nto 
solutions to this problem were suggested in the course of the research. 
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THE MULTIPLICITY OF INFORMATION SERVICES: 
OVERLAP, DUPLICATION , COMPETITION , OR COOPERATION? 

* 

Introduction 

A catalog we developed in an earlier phase of this study identified 157 
organizations that deal with human services information (Applied Management 
Sciences, Inc., and Cuadra Associates, Inc,, 1980:i). Given this number and the 
possibility that new legislation could lead to the establishment of still more 
government-sponsored information service organizations, questions are being asked 
about the need for all of these services. There is concern that the government may • 
be providing services that are or could b6 provided by the private sector and that 
there may be undesirable duplication of effort among government-supported 
information services. Three central questions emerge from this discussion. 

• What role should the government play in providing^ information 
ser^''ces? *^ 

• To What degree do government-supported services compete 
with or pre-empt privately supported information services? 

• To the extent that there is a foie for government-supported 
services, is the present structure, with a large number of 
highly specific information services, the best one for providing 
these services? 

This chapter reviews the issues being considered with respect to each of these 
questions. It identities problems that are perceived to result from the multiplicity 
of information services and discusses possible alternative approaches to service 
delivery. 

Definition of Key Terms 

Considerable confusion exists about issues in part t^ause of terminological 
confusion about what the terms "overlap,'' "duplication,'' aniJ "competition" mean. 
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Several distinct problems are noted in the literature, but the language used to 
describe them is often vety loose. To clarify and separate the issues, we will use 
these terms in specific ways* 

* • "Overlap" will refer to the existence .of multiple 
government-supported sources of information on the same or 
highly related topics. The concept implies encroachment of 
one information service into the topical jurisdiction of 
another. 

• "Duplicatiorf' will be used to describe performance of the - 
same task by multiple government-supported information 
services. It m^y ocqur in designing information services or 
databases, indexing and abstracting, provision of bibliographic 
access, and/or document delivery. The concern^ is for 
repetitive performance of a function rather than rmiltipi'e 
coverage of the same-jHib ject matter. ^ """"^^ 

. . • ' ^ 

• "Competition^ is the term we will use to describe situations in 

which there is' a potential for or there already are both 
publicly and privately supported information services. There * 
may be both overlap and duplication between the services, but 
the primary concern is that government activity pre-empts or 
precludes successful private activity in the field. 

To clarify the distinction between overlap and duplication, consider the 
following illustration. Two clearinghouses in different subject areas scan the same 
journal for relevant articles* From a particular issife each clearinghouse selects the 
following articles for its database: 

Clearinghouse 1 Selects Clearinghouse 2 Selects 

Article 1 Article 1 

Article 2 Article 3 

Article 5 " Article 7 

The presence of Article 1 in both databases probably represents overlap between the 
databases. The scanning of the journal by both clearinghouses and multiple 
development of a bibliographic reference to Article 1 probably represents 
duplication of effort. It should be recognized that overlap between the two 
clearinghouses may be justified by the fjertinence of an article to both topical areas 
or target audiences and that the journal itself may b^ sufficiently generic to justify 

scanning it for both topical areas. The question becomes whether there is a more 

\ ' 

efficient way to perform the scanning task. 

The reason for sayini "probabl/* is that a given article or other item of 
literature can be viewed ana indexed frcrm a variety of perspectives. Thus an article 
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about an experiment on hallucinagenic drugs, using college students, can lend itself 
to the production of a variety' of quite different abstracts oriented toward 
experimental technique, the pharmacological properties of a -particular drug, the 
biologicai impacts of drugs, the personality of college students, or ethical standards 
in research on humans* Even the sets of index terms produced for these orientations 
can be very different from each other. Therefore, while the coverage by different 
clearinghouses of the same set, of articles very likely involves overlap (as well as 
duplication), it does not always, or necessarily, do^so. 

The existence of "competition" is also difficult to determine, with precision. 
There arfe very few identical information products and services and, even where 
there are similar information products and services, one cannot easily determine the 
extent to which the existence of one impacts on the sale or use of the other. Some 
members of a given target, audience, or market, do not know about one (or both) of a 
pair of similar products and services; therefore, it is not justifiajble to claim that one 
of the products or services deprives the other, fairly or otherwise,- of a market. 
These considerations are very important to keep in mind in discussing competition, 
which is often more potential than real. 

The concern for overlap, competition, and duplication covers all facets of 
information service, including acquisition and storage of information sources, 
development of tools for retrieving information, publications development, and user 
services. To limit the scope of this analysis, we have elected to examine '•;verlap 
and duplication in two of the major user services activities: provision of 
bibliographic access and document delivery. These areas -were selected because 
they have been discussed extensively in the government and are topics of concern to 
policymakers. The role of clearinghouses in publications development, another 
major activity, is described in Chapter 5. 

• "Bibliographic access" is the process by which a user is 
informed of the existence of materials of a particular type or 
on a specific topic. Online databases, topical bibliog'^aphies, 
and index and abstract journals are me*;hods for providing 
bibliographical access. 

• "Document delivery" is provision to the user of a copy of an 
item such as a book, journal article, or report in either 
hardform or microform. It may involve sale, loan, or library 
access to a copy of the text. I 
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^ tliHe>Govemment*3 Roie in Information Services 

Underlying \he concerns about competition, overlap, and duplication is a 
broader question: what role should the government assume in providing information 
. *' services? This question has been raised with increasing frequency during the past 
ten years and is currently the subject of a National Commission on Libraries and 
Jnfdrmation Science (NCLIS) TasT< Force, as well as a subject of study by the Office 
of Management and Budget and the General Accounting Office* It was also the 
central theme of 'the Spring 1981 meeting of the Association of Information and 
Dissemination Centers (ASIDIC), It is important to recognize that the question 
pertains to all types of government-supported information services, including 
information analysis centers, GPO, NJIS, and depository libraries, in addition to 
information clearinghouses. 

the Federal government has been involved in the, transfer of information 
virtually since its inception. Early activities included the constitlitionally-mandated 
■ decennial census and the 1790 Patent Act, which established a government 
committee of experts to examine and approve patent applications. To ^support this 
work, the Patent Office developed 'a collection of literature on inventions and 
equipment that permitted it to determine whether patent applications were for truly 
novel items. The Library of Congress, established in 1800, and the National Library 
of Medicine, founded in 1836, are other early Federal information initiatives. The 
Office of the Superintendent of Documents, with\responsibility for the sale and 
» distribution of government publications, was established by the Government Printing 
Act in 1895. These activities and later government invo/.V'./nent in information 
services reflect the view that it is a legitimate government fu;iction to provide its 

citizens with access to information. 

>■ < 

Questions posed in the literature about the appropriate role of government 
involve all phases of information generation and dissemination* These questions 
relate to degree of government involvement and the relationship between public and 
pjuvate services. Questions that are now the focus of national debate include: 

• Is it a government responsibility to publish the results of 
government-sponsored research? 

• Should there be a single source for processing and distributing 
the results of government- funded research? 
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• Is it a government responsibility to facilitate access to its 
publications through the developnnent , of information and 
retrieval services or other dissemination mechanisms? 

• Is there a f^eecl for a consistent national policy with respect to 
information dissemination or should the policy vary by type of 
information and/or type of user? 

Answers to these questions depend largely on one's point of view. Advocates of 
the free market and private enterprise tend ,to favor limited government activity 
and greater reliance on private service providers. They assert that if there is 
sufficient interest in a particular topic, private entrepreneurs will enter the field 
and offer services to meet those needs. Advocates of greater government 
participation in information services suggest that private enterprise cannot be relied 
upon to provide all of the services needed by the public because the ^jrivate sector 
will enter the market only when the demand appears sufficient to produce a profit. 
They maintain that the idea of providing information publications and services on 
highly technical topics and topics pertaining to health and welfare -is not likely to 
attract private enterprise even though it may represent an important government 
priority^ Free-market advocates counter that if the demand is so small, the activity 
may not be a justifiable government venture, either. 

The differences in point of view are largely philosophical, and there are no 
"right" or "wrongf' answers. Berninger and Adkinson, in a study of the interaction 
between the public and private sectors in the delivery of information services, 
suggest that there are no uniquely public or uniquely private information activities. 
They indicate that both sectors perform similar functions, including development of 
primairy and secondary journals and .ewsletters, database development,Hndexing and 
abstracting services, selective dissemination of information, information search 
services, and user education programs. Both sectors also use similar technologies, 
cover similar topics, 'and attempt to serve similar populations (Berninger and 

Adkinson, 1978:14-15). \ 

1 

Another justification for government involvement in information services is the 
belief that there is a qualitative difference between what government agencies and 
the private sector will provide. For instance, an academic association may have a 
narrower view of scientific validity than a government agency and may, therefore, 
be more restrictive in ' selecting items to announce or distribute. Other 
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organizations may De interested in presenting a particular perspective (e.g^ the 
American Cancer Society's and the tobacco industry's views on. the relationship 
between smoking and health). It is claimed that government information services 
are needed as 'a balanced and objective source of information. Our research 
indicates that •this view is not always accurate* Interviews with personnel at 27 
human services information clearinghouses indicate that a majority take a neutral 
stance in their information programs. However, 10 respondents indicated that the 
clearinghouse wa^ established to promote a particular point of view, such as 
educational equity for women, and that this view was reflected in clearinghouse 
publications* In most cases, neutrality was' maintained in the acquisition and 
announcement of literature, but a few respondents indicated that material 
presenting a view that contradicted the purpose of the clearinghouse might be 
excluded from the collection (Applied Management Sciences, Inc., and Cuad^ 
Associates, Inc., January 1981: 3«3-3.4). 

A concern that 0MB has raised recently nas to do with the extent of 
government involvement in information collection, as well as dissemination* Doubts 
are being expressed about the need for the large volume of information currently 
collected and made available through the government* To reduce the volume of 
information being collected, Congress passed the Paperwork Reduction Act of 1980 
with the goals of: 

• minimizing the Federal paperwork burden on respondent^ 

• minimizing the costs to government for collecting, 
maintaining, using, and disseminating informatiof^ 

-maxim izing-the-usefulnes3-of-infofmation-that-i3-collect«c^ and- — 

• coordinating information practices and policies (P.L. 96-5il; 
Section 3501, 1980)* 

The law establishes an Office of Information and Regulatory Affairs within 0MB to 
develop and implement stan ^rds and guidelines for information collection activities 
and interagency sharing cf information* Although the law specifically addresses 
only information collected for use by government agencies, 0MB is interpreting the 
law broadly as a mandate to examine the role of government in dissemjnating 
information* One of the questions being haised is whether all the information that is 
produced is necessary. If it is needed, then questions must be asked about how to 
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disseminate it effectively. Whether the task c\n be assumed by private entities and 
the relative effectiveness of small, dei:entrali2edy government-supported 
dissemination centers targeted to specific fields of interest as compared to larger, 
more general services, such as the Librai^y'of Medicine, are key questions. Current 
thinking en these issues js presented ijcffhe remainder of this chapter. 

Competition Between Government-Supported and Private Information Services 

Discussions of competition- and overlap between government-supported 
/information services and^ those in the private sector tend to focus on the 
inconsistency ^between a government policy of encouraging private initiative and a 
practice of government pre-emptiorv of potential private initiatives. Government 
policy, as Expressed in 0MB Circular A-76, is to rely on private sector goods and 
services where t^iey are available at a reasonable cost, this policy is based on the 
assumption that government Should not compete with its citizens, particularly when 
the private sector can offer the service at an equal or lower cost than the 
equivalent government activity (0MB Circular A-76, 1979:1-2). , The circular 
specifically identifies distribution, research and development support services, 
library operations, and cataloging as information-related activities that cpuld be 
performed by private organizations. 

From the perspective of private entities providing or considering initiating 
information services, the possible entry of a government-subsidized service is 
regarded as a major disincentive to private investment. Because 
government-subsidized services can be provided free of charge or at a fee below the 
^uli costof the operation, they can be offeied at prices that private services cannot 
match. If the services are roughly equivalent, users are likely to select the less 
expensive alternative, making it difficult for private services to gain or retain 
marked. Representatives of the private sector indicate that the existence of 
government-subsidized services precludes the successful operation of private sector 
services. They also assert that there is a threat, even when the private service 
already exists^ because ^ government agencies have initiated competing 
government:^8tipported services without first exploring the potential for existing 
private services to meet their needs. 

Counter-arguments are made by proponents of government-supported 
information services. They maintain that the government* becomes involved 
primarily ip areas where commercial viability is questionable because of the limited 
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market for highly spedific technical information or because of the inability of users 
to pay the full cost of the service. They assert that private enterprise would not be 
willing to serve the target audiences served by go verrment- funded clearinghouses* 
The application of these concerns to*^ document delivery and 'the provision of 
bibliographic access is presented on the following pages* 

Competition in the Provision of Bibliographic Access 

In addition to the many databases being produced by government-supported 
information services, there are numerous databases which have been developed by 
commercial and norvprofit organizations. We spoke with representatives of several 
of these databases, as well as a representative of the Information Industry 
Association, to explore public/private competition in the provision of bibliographic 
access* The vie^ws of those respondents, supplemented by information contained in 
the literature, are presented below. 

Representatives of private information services are concerned primarily about 
the government's pre-empting their market through development of subsidized 
government databases. The concern becomes most acute when a 
government-supported, computer-readable database is made available to the public 
for online searching. The database is then in direct competition with any privately 
developed database in the same field. Private sector database developers are 
becoming more sensitive to any government-supported database actiVfty, however, 
because they see the possibility that many internal clearinghouse databases could 
eventually be made commercially available for online searching. Several examples 
of such a development already dan be cited. 

There are two. separate issues here . The first issue is whether the government 
should produce databases and make the tapes available. The second is whether the 
government VwTd provide online access to the databases through its own computers 
or computer time rented from the private sector. Members of the private sector 
are concerned primarily about direct online access from government-sponsored 
computers. Many private-sector organizations are in favor of having the 
government collect basic data and n;ake it available, on tape, lo all comers, 
including themselves. Private sector database developers, however, are concerned 
about government competition in database development, as well 
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The National Clearinghouse for Mental Health Information (NCMHI) is an 
exan.ple of the latter situation. The NCMHI database,^ funded by the government, 
'was made publicly available through a commercial vendor, BRS, Inc. It contains 
citations and abstracts of the mental health literature, covering both biomedical and 
social aspects of mental health. The database includes materials from journals,, 
books, technical reports, and confer§nces; its coverage is international back to 
1969. The American Psychological Association (APA) produces a database known as 
"Psychological Abstract^* that is also publicly available through BRS, Inc., as well as 
other commercial onliae service vendors. The latter database covers primarily 
journal literature in psychology and the^ behavioral sciences. This privately 
sponsored database has been in existence since 1967 *and is also international in its 
coverage. 

"Psychological' Abstracts'* became publicly available online in 1972 through 
DIALOG 'and latefr became available through other commercial vendors. The 
NCMHI database/ was not publicly available ^r online searching by users until 
considerably latter, although the clearinghouse did perform searches of the database 
for government agencies and their contractors in the interim years. The major point 
to t^^ made that the privately developed database was In existence and publicly 
available for/ about eight years before the government-sponsored database was made 
equally accessible. 

APA perceives the NCMHI database as a substantial competitor that has 
reduced the usage of the "Psychological Abstracts" (PA) database. It appears that 
users may be searching the NCMHI database first and then refining their search 
strategy b^^fore searching the PA files because the usage charges are much lower for 
the NCMHI database. BRS, Iric, can offer the NCMHI database at a lower price 
than "Psychological Abstracts'* because the government does hot charge a royalty 
for the use of its NCMHI tapes* nor depend upon such income to finance continued 
production of the file. 

Private database developers foresee se^^eral problems with this type of 
competition. First, if the private database producer is forced out of business by 
government competition, the government becomes the only available source of 



* The government typically does not require royalties for online use of its 
databases. An exception is NTIS, which levies royalty charges based on the 
amount of database use, in addition to an annual license fee. 
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information in the area ^iffected. This circumstance is not consistent with the 
concept of a democratic free flow of information/ In addition, there is the 
possibility that if government priorities subsequently change, the government 
database may be eliminated, leaving no source of information* Recent discussions 
about closing several government clearinghouses^ including NCMHI, make this 
comment particularly pertinent* 

Although the major competition^ arises when ^ the government-supported 
clearinghouse databases become publicly available, some private database producers 
question the necessity for developing any clearinghouse-specific databases. Several 
producers suggest that the government first solicit the existing database producers 
about their willingness tc expand an existing database or establish another related 
one -before embarking on a new government-sponsored venture. They suggest that 
they would be cooperative if a real market were known to exist. The Department of 
Commerce, under pressure^from the Information Industry Association and others, is 
following such a path in developing WITS, a Worldwide Information Trade System 
designed to provide information on domestic producers and their products and the 
needs of overseas buyers* 

y 

We explored the willingness of private database producers to participate in 
cdoperative ventures with the government as a means of moving from competition 
toward greater cooperation between the two sectors. The Women's Educational 
Equity Communications Network (WEECN) tertiary database and the WITS example 
offer some basis for optimisnr), but there still are problems that must be overcome. 
Some publishers and database producers are reluctant to beconrie^ involved in any 
enterprise that involves working with a government agency. Irythe 1960s, when the 
ERIC system was planning its index and abstract publication of the jpurnal 
literature. Current Index to Journals in Education (CUE), the publisher of the 
Education Index was approached about the possibility of expanding its operation^ to 
meet the needs of the ERIC system. At least one reason cited for the^Wilson 
Company^s refusal was hesitancy about getting involved with government agencies 
and subsidies* 

Another problem from the perspective of the private database producers is the 
potential loss of copyright protection on any citation or abstract that is picked up 
for use in a government-supported database. Government information is not 

/ 
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protected by copyright and producers of copyrighted databases are concerned that 
any citations and abstracts thrt become included in a ^vernment database are no 
longer protected* They might be willing^to permit such inclusion if the lease 
provisions include a copyright protection or if the government database simply 
referenced the relevant items in the private database without including the 
connplete abstract* Private database producers prefer the second alternative 
because they are reluctant to lose control over the content of the citation and fear 
that the government might alter or reformat the citation for use in its database^^^f 
the first alternative were implemented* 

An alternative to creating government databases is a partial subsidy to 
encourage private sector development when the government perceives the need for 
a new database* Several respondents indicate that either tax incentives or matching 
grants could be used to stimulate private database producers to eihter new^3u5ject 
areas* Such subsidies would reduce the risk involved in entering an untested market* . 

Competition in Documeht Delivery 

The role the government chooses to play, eilhjefr through a clearinghouse or • 
some other mechanism in delivering documents reporting on government-sponsored 
research, determines whether there is any pojt'ential for private organizations to 
become involved in document delivery* The/government may act as the primary 
distributor, as wholesaler, or as a facilitator supporting the creation of private 
distribution mechanisms* When a clear;dghouse serves as the primary distributor, it 
reproduces copies pf government-spdnjsored research reports and^^distributes them, 
upon request, free or for a fee* In the role of wholesaler, the clearinghouse may sell 
or distribute publications in bulk to intermediaries who in turn distribute them to 
users* As a facilitator, ^he clearinghouse's role is one of making the document 
available to a vendor, sdch as NTIS or a commercial publisher, for reproduction and 
distribution* 

Advocates of greater private sector involvement assert that the private sector 
can efficiently provide document delivery services and that there is no need for 
government-sponsored clearinghouse^ to perform this function* The problem, as 
they view it, is not so much one of duplication of effort but rather that the 
government has sometimes pre-empted the market and precluded the potential for 
commercial ventures* 
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An example of 'government competition involves the Congressional Information 
Service (CIS), a private company that has been collecting, abstracting, indexing, and 
microfilming Congressional documents' for sale to the public for over ten^ years. 
Recently, GPO began microfilming the same documents for distribution to the 
depository libraries. GPO previously printed these materials in hardform only, but 
moved toward microfilming as a cost-saving measure. CIS is worried that once GPO 
has begun /microfilming Congressional documents, it will also begin selling these 
documents in microform to the general public. CIS considers such a development an 
infringement on its market and contrary to the policy expressed in 0MB Circular 
A-76. 

The ERIC Document Reproduction Service (EDRS) illustrates a possible 
approach to cooperation between a government-supported clearinghouse and the 
private sector in document delivery. ERIC Central enters into a five-year 
Agreement with a commercial organization to reproduce and distribute copies of 
documents collected by the 16 ERIC- clearingh. ^es. The contract is competed 
every five years and is awarded to the bidder who can provide service to users at the 
best price. The contractor .is supported by the sale of copies directly to users and 
retains all fees that are collected. * A similar arrangement exists between the 
Securities and Exchange Commission and Disclosure, Inc.,'for the sale of mandatory 
filings by publicly owned companies that trade on the New York, American, or 
over-the-counter stock exchanges. ^ 

Although such arrangements allow the private sector to become involved in 
document delivery, some representatives of the private sector maintain that this 
^tlxictuTe stTirpeTmits~td fhuch government interference in the marketplace. By 
granting a five-year contract to one organization, the government may adversely . 
affect other firms that wish to enter the market. These critics prefer a"hands-off' 
approach in which no organization is given a competitive edge. 

Another facet of the document delivery to »ic relates to providing the text of 
non-government publications, such as journal articles or sections of commercially 
published, books* Some publishers complain about infringement of copyright 
protections when portions of their journals or books are reproduced for distribution 
in resppnse to user requests for information. They assert that reproduction of 
selected articles reduces the market for the sale of their publications. The new 




copyright law statoS that when "systematic" photocopying of copyrighted materials 
occurs, the authorization of the copyright owner must be obtained each time a copy 
is made. Although the language of the law focuses on libraries, it applies equally to 

other types of information services, including clearinghouses. 

> 

" Data collected for this study indicate that of the clearinghouses in the sample 
that provide document-delivery services, over one-third limit the service to 
non-copyrighted items. Those clearinghouses that reproduce copyrighted items 
usually provide items that the clearinghouse itself has produced or items for which 
the author has signed a release. Other methods, used by one clearinghouse each, 
include interlibrary loan of copyrighted materials and pxovision of no more than one 
copy per request of any copyrighted item. 

Overlap and Duplication Among Government-Supported Information Services 

If one accepts the idea that there is an important role for government in the 
dissemination of information, questions still remain about the most efficient and 
effective structure for performing these activities. The present information 
dissemination structure of the Federal government includes many small, specialized 
information services targeted to the needs of specific audiences. Specialization 
occurs through a focus on rather narrowly defined subject areas, such as health 
indexes or women's educational equity, or through selection of a narrowly defined 
target audience, such as researchers or patient educators. Specialization has led to 
a situation in which there may be several services offering information on related 
topics (e.g., a clearinghouse on child abuse and neglect and another on domestic 
violence among adultsX of several services offering information on the same topic to 
different audiences (e.g., separate cancer information services for researchers, 
patient educators, and the lay public). 

In addition to the small, specialized services— frequently called 
clearinghouses— the government also supports several broadly focused information 
activities, some of which produce large and comprehensive databases. The most 
relevant^ ones for human services information ar the MEDLARS database of 
biomedical information compiled by the National Library of Medicine (NLM) and the 
ERIC database of education information, sponsored by the National Institute of 
Educatiuru 
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The existence af both the broad databases and the smaller, specialized 
databases In " government-sponsored clearinghouses raises questions about the 
potential for inefficiency through overlapping collections and duplication of effort 
The t>pes of questions raised about overlap include: 

• To what degree are there .multiple references to the same 
materials? 

• To what extent are the same topics addressed by multiple 
information services? 

• What effect does overlap, among information services heve on 

users? ' 

Questions pertaining to duplication of effort include the following: 

• What is the cost of acquiring and storing the" same or similar 
materals by multiple services? 

• To what extent are multiple services -allocating resources to 
scan, index, and abstract the same literatOfre? 

• Is it necessary to incur development and maintenance costs 
associated with separate databases for each service? 

Concerns about overlap and duplication in provision of bibliographic access and 
in document delivery by government-supported information service^ are discussed on 
the following pages. Techniques that offer the possibility for reducing overlap arid 
duplication are also presented* ^ 

Overlap and Duplication in Providing Bibliographic Access 

The potential problem in pro\7iding bibliographic access involves citations to the 
same publication available in different databases* This sitijation occurs when 
several information service organizations on related topics r lUect and cite the same 
document. There may also be a more limited type of consent overlap wnen the 
organizations cover the same subject matter even though the referenced items are 
different. 

Although assertions about the pervasiveness of overlap in the provision of 
bibliographic access are widespread, there is little evidence on the actual extent of 
the problem. Those studies that have been conducted have generally focused on 
overlap with privately produced databases (Perk, 1977 and Oppenheim and 
Perryman, 1980). The studies indicate some methodological problems that inhibit 
the researchers* ability to assess the extent of overlap. The absence of standaraized 
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document identification formats makes it difficult to determine, in some, cases, 
whether the same item i3 bping referenced (Rowlett, 1971:111). Another problem is 
determining what comprises overlap. For example, several of the same journals are 
scanned by two organizations developing databases in the field of education. 
^However, one database indexes all articles in the journals, whdreas the other is more 
selective in its coverage and scans a broader range of journals (Pe.rk; 1977:222-223). 
Whether this constitutes overlap or simply the existence of two different. products 
serving different audiences depends on one's definition and perspective. Since the 
end products are quite different, one could justifiably argUe that this situation does 
not constitute overlap. Specific approaches to^ measuring overlap that have been 
tested on a limited basis are briefly described in the Afjpendix to this chapter. 

Clearinghouse personnel report that clearinghouse collections frequently 
enriphasize types of materials different from those, in the large comprehensive 
databases. Thur they believe that overlap among the collections is not' as extensive 
as the sim'larity in topical focuses might suggest. 

Duplication in tlje provision of bibliographic access involves multiple 
performance by two or more organizations of the tasks required to store and 
retrieve^ information. The possibility, of multiple investments in (1) computer 
hardware, (2) design bf storage and retrieval systems, <3) and staff time for scanning 
the literature and preparing bibliographic citations raise concerns about inefficient 
utilization of government funds. Policymakers are asking whether it would be 
possible for the organizations that produce the comprehensive databases to perform 
these tasks for all information clearinghouses or whether some coordinative and 
cooperative arrangements could be worked out among the services to share 
equipment, designs, and perhaps even personnel. 

Clearinghouse directors and their Federal Project Officers accept that some 
duplication and overlap in providing bibliographic access occurs and they cite the 
following justifications for this phenomenon: 

• . The clearinghouses need a complete database to support their 
user services activities because it becomes too costly to 
reference multiple external databases continually in response 
to que&tion^ 

n The large databases tend not to be as current as the 
clearinghouse databases, and clearinghouse services would be 
'less up-to-date if they had to wait for materials to be 
announced through the larger databases; and 

i 
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• The index terms and abstracts developed to describe the 
common references differ depending on the needs and 
interests of the audience being addressed^ therefore it is not 
possible for all clearinghouses citing the* same document to use 
a;single set of ihdex terms or a common abstract. ' . 

Failure to explore the feasibility of expanding an existing clearinghouse, before 
deciding to establish a new one, is considered a fairly common occurrence. It often 
results from the pressure to respond to Congressional or agency mandates for an 
information service to be operational within a specified period of time. Project^ 
Officers, in particular, are critical of the limited time often provided to plan a new 
information service. They assert that insufficient time is allowed to explore current 
resources, to negotiate with them, or to learn from their experiences. Instead, the 
emphasis is to get the new service "up and running" as quickly as possible. Pressure 
groups who want their own clearinghouse independent of those already in existence 
also contribute to the absence of sharing and limited efforts to build on what exists. 
Difficulties with combining different funding sources are another impediment to 
cooperation. 

Clearinghouse personnel also point out that overlap is net necessarily wasteful 
and detrimental. They maintain that the large collections, such as the MEDLARS 
collection of the National Library of Medicine, are difficult for users to access 
because of their vast size and research orientation. They claim that subsets of 
these collections respond >to highly specialized target audiences with unique needs. 
The clearinghouse databases, therefore, may include subsets of numerous other 
databases, with abstracts and index terms modified to be relevant to the needs and 
vocabulary of a specific audience. 

Overlap and Duplication in Document Delivery 

Much of the discussion about duplication among government-supported 
information services in document delivery addresses services other than information 
plearinghouses* It has focused on relationships between NTIS and GPO and the 
adequacy of the depository libraries as a substitute for clearinghouses* 

One reason that clearinghouse involvement in document delivery has not 
generated substantial concern may be that not all clearinghouses engage in this 
activity. Among the sample of 27 human services information clearinghouses' we 
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visited, 15 provide document delivery for publications not developed by the 
clearinghouse, four provide limited document delivery services for selected items 
not available elsewhere, and eight do not offer document delivery services. There 
may be occasional duplication in document delivery when an item in the 
clearinghouse collection is also available through NTIS, GPO, or another government 
agency. More commonly, however, the duplication occurs in distribution of 
clearinghouse-generated publications, such as synthesis and analysis publications* 
This type of duplication can occur when GPO performs a print run for a 
clearinghouse and elects to print additional copies for sale through GPO bookstores 
or for distribution to the depository libraries. 

Multiple points of access to a clearinghouse publication should not be regarded 
as a serious- problem. Although some users have complained because the same 
publication is available from different sources at different prices or because they 
resent paying for an item that others receive from another source free of charge, 
this situation cannot be averted if publications .are to be broadly available* It is 
apparent that some duplication is necessary to provide public access. For example, 
libraries in San Francisco and New York "duplicate" each other in prov»iding access 
to the same magazines^ but almost no rational person would imagine removing this 
duplication, because it is riot convenient for a New Yorker to come to San Francisco 
to read a magazine in its library* Similarly, access to a -clearinghouse- generated 
document from multiple sources is more of a convenience than a duplication of 
effort. Also, different sources have credibility among different audiences* 

Current Approaches to Addressing the Problem 

Evidence of cooperation between clearinghouses and with the large 
government-funded databases is presented by clearinghouse personnel to minimize 
OMB's and GAO^s concerns about overlap and duplication. Clearinghouse personnel 
point out that there is a conli^nuous effort to define each organizations scope to 
avoid overlaji among collections. In the eyes of clearinghouse personnel, the 
National Library of Medicine^s MEDLARS database has, been primarily a resource to 
support researchers. (It should be noted, however, that, the NLM database is widely 
used to support the information needs of practitioners and health educators.) 
Personnel at several health clearinghouses report that they fccus on 
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literature thai is more appropriate to practitioners and patient edui^ators- 
Clearinghouse personnel also indicate that when one clearinghouse finds a document 
that is more appropriate to another's focus, the document is typically sent to the 
second clearinghouse for consideration* Sometimes the comprehensive database is 
checked to see whether an item is already listed before the specialized 
clearinghouse decides to acquire and process it* The National Clearinghouse for 
Bilingual F,ducation uses this procedure to reduce overlap with the ERIC system^ for 
example* \ 

Policymakers' concerns about overlap and duplication are particularly relevar^t 
to the health clearinghouses because of the existence of the NLM's MEDLARS 
database and the many separate health information databases that support the^ 
clearinghouses* The ERIC system, with its 16 specialized clearinghouses, produces a 
single database for the announcement of education-related materials. There is some 
debate about whether the ERIC model could be successfully applied to the health 
field. ^ One clearinghouse director stated that if DHHS were to set the standards, all 
health clearinghouses could follow tne same indexing and abstracting format* Other 
clearinghouse directors are less inclined to follow the ERIC model. Some of them 
cite as evidence of the failure of this model the fact that new clearinghouses in the 
education field are now being established outside the ERIC system; however ERIC 
personnel explain this phenomenon in a different way. 

ERIC personnel explain the move toward the establishment of cloaringhouses 
outside the ERIC system as a'' result of representatives of new interests in the 
education community wanting clearinghouses in their fields to provide more 
extensive services to users. Once they receive funding, these organizations prefer 
to establish a totally independent entity* ERIC personnel suggest that another 
approach might be feasible and more economicaL They indicate a willingness to add 
new subjects to the ERIC database if the interested organizations can provide funds 
to support that activity* Such a process would eliminate the costs incurred in 
developing a new database and wouJd provide coverage within the frame vork of the 
existing ERIC database* The interested organizations could still fund their own 
information ^ctivities to provide other clearinghouse^ services, such as personalized 
reference services, searches of the database, development of networks, and other 
activities that are not a niajor function of the ERIC system. 
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The possibility of consolidation and more coordinated control among health 
clearinghouses has recently been studied by the Public Health Service and the Office 
of Human Development Services. The Public Health Service study (Comstock and 
Pollack, 1981) was initiated in response to the apparent lack of coordination in 
establishing and operating health clearinghouses and the Public Health Service's 
interest in determining whether any economies of scale could be achieved through 
greater cooperation* Six management alternatives, ranging from maintaining the 
current* approach to total consolidation into a single health clearinghouse, were 
considered. Retaining the present structure was rejected because of the known 
duplication of effort in databases and software development. Total consolidation 
was rejected because it would be difficult and extremely time-cohsuming to gain 
cooperation for dismantling existing operations and developing compatible systems 
for meshing ongoing activities. The report recommends an intermediate approach 
involving establishment of a Division of Clearinghouse Management to assume 
consolidated responsibility for major clearinghouse decisions (e.g., establishing or 
closing a clearinghouse, purchasing equipment), while maintaining oversight of daily 
operations at the (irogram level. A task fcrce has now been established by the 
Assistant Secretary for Health to consider alternatives and plan for the 
implementation of a coordinative mechanism for Public Health Service- funded 
clearinghouses. 

This alternative reflects awareness of political realities that are quite different 
from those that existed at the time the ERIC system was established. At that time, 
no education clearinghouses were in operation and no direct clearinghouse/program 
sponsor relationships had been established, fn contrast, today these relationships 
exist in the health field and a.re strongly supported by operaUng clearinghouses and 
their sponsors. The approach proposed by the Public Health Service does not involve 
consoli,dating database development or establishing a single contact point for 
communicatirtg with target audiences. The repqrt recognizes 'that, although 
economies of snale might be realized if these activities were consplidatsd, such a 
union would not likely occur without strong support from the highest levels of DHHS. 

The Office of Human Development Services (OHDS) has alsc conducted a recent 
study on the potential advantages of greater consolidation of information 
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services and duplication between public and private services (U*S* Department of 
Health and Human Services, July 1981)* The study was conducted in a very brief 
period of time and does not claim to be a detailed analyses of ail the issues* The 
report Jndicates that there. is duplication among government services and between 
the public and private sectors in the types of publicacions and services provided and 
concludes that there (night be some advantages to be gained from greater 
consolidation* Because of the proposed elimination or reduction of several OHOS 
information programs, the- report does not recommend any^ immediate action but 
suggests the need for further study* 

From a management perspective, it is important to decide whether the 
economies of scale that would potentially be realized outweigh the costs involved in 
achieving that consolidation* The costs .^would include antagonism from 
organizations or program offices that currently sponsor Clearinghouses, resistance 
from clearinghouse personnel and their constituencies, and the operational problems 
bf meshing currently incompatible systems. The possible benefits include building 
on the knowlfidge and capabilities that already exist within the government to 
process, announce, and distribute materials; elimination of activities that "reinvent 
the wheel"; and reduction of multiple processing of the same materials. 

Another approach to cooperation is the development of a tertiary database, a 
model tested for two years by the Women's Educational Equity Communications 
Network (WEECN)* The concept involves developing a highly specialized database 
from^ citations in existing secondary sources (i*e*, other databases)* WEECN 
searched approximately 13 relevant databases cn a regular basis to extract pertinent 
references. Separate agreements were negoti5''.ed with each source database to 
permit 'extraction of the citations and their reformating to a WEECN style* 
Creators of the proprietary databases were paid a royalty for each citation selected 
from their database* When WEECNI identified a document that was pertinent to its 
collection but not currently cited in any ot the source databases, it forwarded che 
document tb the most appropriate database producer* WEECN then picked' up the 
citation when it was entered into that producer's database* 
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APPENDIX 

Techniques for Measuring Overlap 
among Databases 



Several studies have suggested different approaches to measuring overlap 
among databases^ A summary of the techniques is presented below. ' ^ 

Oppenheim and Perry man (1980) suggest the use of a relative recall measur^ to 
assess overlap between two databases* The formula they use lis: 



// of relevant items recalled 
in database A 

// of unduplicated relevant 
items recalled in databases 
A&B 



if of relevant items recalled 
in database B 

// of unduplicated relevant 
items recalled in databases 
A&B 



If there is complete overlap between the two databases, the sum of the two 
equations will be 2.00. If there is no overlap between the two databases, the 
sum of the equations will be 1.00. Any result between 1.00 and 2.00 identifies a 
proportion of overlap. For example, if there' were two databases, one with six 
relevant references and one with four relevant databases, and no common 
citations, the results wouW be 

6 * '0 inn 

4+6 4+6 10 

If there were two common citations, the results would be 



T 8 - 8 ■ 



or 25 percent overlap. 
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The term "notional overlap" has been used tp describe overlap in content 
between two different documents* This concept can be applied to measuring 
the* degree to which clearinghouse databases cover the same subject matter, 
although not necessarily^ the same documents. A study by the Bath University 
Library (1973) suggests that notional overlap can be measured by looking at the 
number of common citations among a set of documents* They call this 
approach "bibliographic coupling*" 

Cleverdon and Kidd (1976) suggest a method for measuring notional overlap 
within a collection or between collections using the same index terms. They, 
suggest that most databases assign seven to ten index terms to a citation* By 
conducting a search for related items using only three index terms, one can 
measure the extent of match on the remaining "free" terms (i.e., the terms not 
used in the search)* If a prescribed threshhold number of matching index terms 
is set in advance, it is passible to assert that all matches in excess of the 
threshhold level represent documents whose content overlaps* The overlap 
score when examining two documents can be derived with the following formula: 

matched terms above threshold level 
~ // of free terms 

F or comparing subsets of a collection or two collections using the same 
indexing vocabulary, the formula would be: * - • 

all matches above threshold ' ^ total // of index terms 

// of documents . (// of document's- 1) average # of free terms 

2"" " per document 
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, • • CLEARINGHOUSE PUBLICATIONS PROGRAMS 

Introduction ^ 

Federally* funded clearinghouses produce and publish a wide variety of 
publications, including both primary publications (e«g«, state-of*the-art reports and 
other original literature) and secondary publications (e.g., -indexes and 
bibliographies). In the context of current government priorities, questions are being 
raised about the necessity of these and other^^government produced publications. 
The pricing policies and dissemination strategies for publications are also beihg 
reconsidered. This chspter examines current publication programs in Federally 
funded clearinghouses in light of these concerns and explores the options available 
for future publication activities. Specifically, the following topics are addressed: 

t the historical context in which clearinghouse publications 
programs have developed^ 

• current government views toward clearinghouse publications 
programs; 

• current clearinghouse publications programs; 

' % planning' considerations for future publications development; 
and 

♦ 

t \ management considerations in response to government 
priorities. 

The Historical Context in Which Clearinghouse Publications Programs Have 
Developed 

. ^ Before 1960, when the concepts of information centers and clearinghouses were 
in their developmental stages, the functions- typically associated with these 
information programs were limited primarily to document acquisition, indexing and 
abstracting, and document delivery. The publications resulting from these activities 
were index .and abstract journals and other bibliographic publications documenting 



the available literature in a given field (Applied Management Sciences,Inc. and 
Cuadra Associates, Inc., 1980:11-14). In 1963, the Panel on Scientific Information of 
the President's Science Advisory Committee issued the "Weinberg Report," which 
suggested a shift in these functions by emphasizing the need for analysis services. 
Stating that "retrieval of documents is not Ihc same as retrieval of information," 
(President's Science Advisory Committee, 1963:32) this report called for information 
centers to review, synthesize, and interpret the published literature to provide users 
with information, rather than source documents. It was suggested that syntheses 
would better maet the users' needs by presenting information in a format more' 
oriented to the application of research. This perspective led to the addition of 
information analysis products to the emerging concept of the clearinghouse 
publications program. As an example, soan after the first ERIC clearinghouses were 
established in 1966, the need for information analysis products was redognized. The 
development and distribution of these types of publications is now 6 central function 
in-the ERIC clearinghouses (Trester, 1979:283-286). . 

The need for information analysis products was further specified in the 1^969 
"SATCOM Report,'* prepared by the Committee on Scientific and Technical 
Communication of the National Academy of Sciences' National Academy of 
Engineering. The "SATCOM Report" emphasized the special information needs and 
rights of practitioners, in contrast to those of scientists and technologists. Making, 
the distinction between research-oriented literature and practitioner-oriented 
literature, the "SATCOM Report" stressed the need for information services to 
collect and synthesize literature of interest to the practitioner. 

Serving the information needs of the human services practitioner and 
policymaker took on increased importance with the growth of^"problen>oriented" 
human services programs* during the 19608. The information available to meet the 
needs of practitioners and policymakers in these, programs typically spanned several 
scientific disciplines and was highly technical. Information service organizations, 
such as clearinghouses, were considered a means for synthesizing and repackaging 
discipline-based literature across several different fields to meet the specialized 



* For a more detailed discussion of. the "problenvoriented" programs and 
information services developed during the 1960s, see Applied Management 
Sciences, inc; and Cuadra Associates, Inc., 1980:4- 14. 



needs of these audiences • (Applied Management Sciences, Inc.. and Cuadra 
Associates, Inc., 1960:5*14)« To disseminate information in a format that is relevant 
to. the needs of their users, clearinghouses review available, irfformation, analyze it, 
and repackage it into new formats, that facilitate the users' awareness and 
understanding of knowledge. Consequently, some olearihghouse publications 
programs have expanded beyond bebliographies and index and abstract journals to 
include such publications as monographs, literature reviews, state-of*the<-art papers, 
and handbooks. 

Current Government Views Toward Clearinghouse Publications Programs 

, The political environment, which once provided support for clearinghouses' 
information analysis and dissemination functions, has shifted recently to a position 
that is mors restraining. The appropriateness of the Federal government's 
involvement in information dissemination and its efficiency in managing this 
dissemination are now being examined. In today's political environment, emphasis is 
being placed on reducing the amount of money spent on government-produced 
publications and on developing better management for government dissemination 
activities. In this environment ^ '^e is a growing belief that government 
publications should not infringe on commercial and non-profit publishing and 
information dissemination efforts. This philosojJhy is forcing a revie^ of the 
publications programs of all government agencies, including clearinghouses. 

Since 1978 several reports have been issued indicating growing concern about 
government publications, including those of clearinghouse programs. One of the 
first indications of this concern was a 1978 survey conducted for the Senate 
Committee on Governmental Affairs. This survey collected data on the volume, 
costs, and distribution of government , publications in 80 department? and agencies. 
Results indicated that even basic information on Federal publications could not be 
adequately identified, and that distribution procedures were inconsistent and 
confusing to the individual requester (Library of Congress, 1979:51-91). 

A second report, xJocumenting hearings held by the Joint Committee on Printing 
.(JCP) in, 1978^ identified major issues and policy questions, explored options, and 
developed cfuestions concerning the government's system of printing and distributing 
information, as related to revisions to Title 44 of the U.S. Code. The Committee 
determined that the role of the private sector in distributing • government 



information needed to be clarified. Private publishing of government information 
was discussed as a way to minimize overlap and competition between the public and 
private' sectors. It was suggested that a system be established to provide 
Qovernment information to private publishers, without the JCP approval for each 
request that is currently required; 

The Office of Management and Budget's Bulletin No. 81-16 placed a 
moratorium on Federal spending for new periodicals, pamphlets under 80 pages in 
length, and audiovisual products, effective April 21, J981. It also called for a 
comprehensive review, which has recently, been conducted, of all existing 
information products, as. well as those planned for the next two years. This review 
was intended to identify ways to reduce Nonessential publications and recover costs 
wherever possible. Agency control systems are currently being established to 
monitor publication activities. 

A draft 0MB circular that has not yet been issued in final form provides 
another indication of 'the government's current perspective. It addresses the 
proliferation of publicly-supported information dissemination activities that 
increase Federal costs, place the government in "unnecessary competition with the 
private sector, and inhibit th^ ability of the private marketplace to provide 
information goods and services" (Office of Management and Budget, 1980:^). It 
proposes that the Federal gover iment place greater reliance on the private sector 
to disseminate public ini^^:*^' .^on. In addition, it calls for each Federal agency to 
establish mechanisms for identifying the costs associated with information 
.dissemination, including costs for printing, processing, and retention of information 
publications. 

Clearly, the Feoeral government is questioning the level of its publication 
activities and current dissemination strategies. Gi ester reliance on the? private 
sector is suggested as a means for continin*ig to disseminate government 
information while at the same time reo «cmg the F3deral buc^get. These concerns 
have oirect implications for clearinghouse publication programs. f-Iowover, before ' 
these conccsfns can be translated into specific management considerations, current 
clearinghouse publications progr'^.ms met be understood. 

Current Clearinghouse Publications Programs 

All clearinghouses produce one or more different types of publications as part 
of their information analyses end dissemination activities. These publications are 



distinct from those referenced in the clearinghouse's database or -heic^. wjchin its 
collection and in some cases reproduce;^ . For distribution upon requestV through 
document delivery services. PubIic?tion3 produced by clearinghouses are those that 
are written, printed, and distributed with clearinghouse funds. In addition, some* 
clearinghouses pri/it and distribute docurnents written through other government 
offices, or through research contracts and demonstration grants, to provide wider 
dissemination .for these documents than vlrould otherwise occur. Clearinghouse 
publications represent "tools" through which most clearinghouses actively carry out 
their dissemination function by synthesizing information to meet tlie needs of their 
users. 

Based on the best available data, publications VoQ^^^rns appear to consumtf ^ 
relatively significent prop^ion of clearinghouses' operating budget?.^ Ohct. 
coflected from a sample of 27 clearinghouses indicate that a mean of 15 perc'ent (a 
median of 11 percent) of total cisaringhouse operating costs is allocated to the 
publication development function. An additional mean of 14 percent (a median of 7 
percent) is allocated to printing and a mean of 3 percent\^ median of 2 percent) is 
allocated for mailing costs. These ^<gures suggest, th-it a mean of 32 percent (9 
median of 20 percent) of clearinghouse operating costc/are budqete^ for puolications 
programs (Appiied Management Sciences, Inc., and C^jadra Associates, Inr., 
1981:7.15).* A less yeneralizable indication of the budgetary significance of 
publication programs can be seen in the ERIC clearinghouses. Within the ERIC 
system approxii.iately one third of each clearinghouse's total budget has been spent 
on information analysis publications (Trester, 1975:294). 

Clearinghouse publications span a wide range of document types. The most 
typical combination of publication types produced by clearinghouses includes 
bibliographies, newsletters, original literature, and index and abstract publications. 



* It must be noted that these figures represent estimated expenditures of only 
those clearinghouses in the sample that could provide budget data. Informati9n 
provided was often difficult to separate out into comparable functional 
categories. Additionally, the three percent allocated to cover mailing costs 
may be used to mail materials other than*clearinghouse pubJications and the 14 
percent apportioned for printing may cover printing beyond that required for 
clearinghouse publications. Given the nature of the data, this assessment can 
only be interpreted as the best t^nown indication of the relative significance 
that pubiicatidns programs play in the budgets of most clearinghouses. 
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Less commonly produced publications are reviews, summaries, and miscellaneous 
publications such as promotional' broc^ures and >fact sheets. Almost all 
clearinghouses produce bibliographies. The particular combination of publications a 
clearinghouse produces is determined by the clearinghouses . dissenriination 
objectives, the needs of its target audiences, and its funding level. ^ 

Publications such as newsletters, bibliographies, reviews, anQi original literature 
are typically produced in periodical style witf\ multiple issues or multiple topics 
addressed each year. The number of issues, per periodical, produced each year 
varies most often according to the extent of identified need, the clearinghouse's 
workload, and ayailat e funds. Most clearinghouses rely on program directives, user 
feedback, and/or suggestions from advisory bp'ard members in determining the 
specific topics to be addressed. Although clearinghouses typically maintain an 
awareness of existing publications to prevent duplicating existing products, they do 
not often consider whether a needed publication could be produced by a 
oon-governmental organization. 

Most publications are produced with clearinghouse funds and authored by 
in-house professionals or outside' authors, yyho are contracted or paid an honorarium. 
Outside* authors may be invited or are sometimes chosen through a competitive 
selection process. Publications in draft form are usually critiqued by in-house staff 
and advisory board mem^bers, but not all clearinghouses routinely subject their draft 
publications- to outside review by experts or a sample of users. 

Documents are most often printed through the Governmerrt Printing Office 

(GPO), if more than 25,000 impressions are to be printed. Smaller printing jobs are 

handled by clearinghouse printing facilities or are contracted out to private printing 
« 

companies. Primary distribution is typically provided through the clearinghouse. In 
some cases, publications are mailed to all or selected persons on the clearinghouse 
mailing list without charge. In other cases, the availability of a new clearinghouse 
publication is announced through the newsletter or index journal and distribution 
occurs on request, with or without charge. 

Although clearinghouses as a whole follow these publication procedures, some 
clearinghouses use more innovative arrangements in their publication programs. A 
few clearinghouses arrange to have authors, mostly academicians-, write scientific 
publications on a volunteer basis as an akernative to paying in-house staff or 
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contracted authors to develop publications. This approach is possible* when 
clearinghouses have established working relationships with experts in the field to^ 
whom topics needing attention can be suggested for journal articles or other 
scientific publications. With these arrangementSi clearinghouses are able to initiate 
publications that are written and produced without clearinghouse funds, but they 
then have little control in imposing deadlines or guaranteeing publicatioa. 

Several clearinghouses provide some of their draft* publications to commercial 
publishers or private organizations to be printed and distributed at no cost to the 
government. The best available information suggests that nearly 60 percent of 
government-wide information organizations, including human services information 
clearinghouses, have macje,at least one of their, information products or services 
available forisale by the private sector (King Research,. Ipc, 1980:21-22). \VKen a 
human services clearinghouse makes printing and distribution arrangements with the 
* private sector, the arrangements are most often with a professional association, a 
journal publisherj, or a scientific and technical publishing company. In some cases, 
clearinghouse personnel write journal columns or articles to be printed and 
distributed by the journal publisher. Other clearinghouses have co-publishing 
arrangements whereby the clearinghouse provides * a draft publication to a 
comnnercial publisher for printing and marketing or provides printed copies to^a 
professional organization which assumes the costs of distribution to its members. 

Current clearinghouse publications programs will inevitably bfe altered by the 
changing political Environment discussed earlier. Government directives call for 
specific changes to the level of effort, pricing policies, and dissemination strategies 
that have been used in the past. These specific ^hanges will be discussed more fully 
in later sections of this chapter. However, the increased scrutiny^eing applied to 
government publications programs, in general, suggests the need for stronger 
planning in the process of publication preparation, as well. Potential planning 
procedures are addressed below. . 

Planning Considerations for Future Publioations Development 

The. first and most basic question that must be considered in planning future 
publications is whether previous assumptions in determining the ^>es of 
publications to produce are still applicable. With the broader availability, of online 

/ 
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searching capabilities and the expansions in pc{)fessionai publishing, each publication 
type should be re-examined to determine whether it continues to be the most 
efficient way to serve user needs. For example, index and abstract publications or 
short bibliographies may have been effective v4ien manual searches of materials in a 
coUectiorl were rfiore common* The more widespread use of online databases and 
the quicker and more comprehensive response to user requests that they/allow novi 
suggests less need for these preassembled reference publications. Clearinghouse 
information analysis publications may not be as necessary as they once were because 
of the expansion in specialized professional literature, particularly journals, during 
the ^ past decade. These developments suggest that clearinghou^ses may need to 
perform a thorough review of £he ' assumptions underlying the selection • of 
publications types* The changing environmenf'may require a new set of responses. 

The second factor to consider is the selection of topics to be addressed. In the 
past, a commitment to producing a specific number of publications each year has 
influenced the topic selection process. If publications programs are to r'eceive more 
careful' scrutiny in the future, then a more rigorous needs assessment may be needed 
to justify each new publication. The needs^assessment prociss could include greater 
use of data, on user requests and a review of/v^alternative materials available from 
other sourc§s as justif^ication for selecting topics for clearinghouse publications. 

Th^ third aspect of the planning process that could be strengthened is the 
publication development and review process. 'A mo^e consistent use of competitive 
selection in choosing outside authors and a more thorough review of draft 
publications by professionals and users in the field would help to maxirViize the 
quality and utility of future publications. The Health Message Testing Service of 
the National Institutes of Health has devised formats and procedures for pretesting 
^published materials that could be adapted for use by clearinghouses (U.S. 
Department of Health and Human Services, n.d.). V 

Management Considerations in Response to Government Priorities ^ 

Clearinghouse managers are also faced with the need to re-examine 
clearinghouse publipations programs^ in response to specific administrative directives 
affecting the level of publication effort, pricing policies, and dissemination 
strategies. Management considerations must' take into account the current 
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adm in istra*" ion's emphasis on overall budget reductiohs^and greater reliance^ on the 

% ft 
private sectdr in disseminating government Information* Given J^e current 

administrative emphasis on budget reductions, allocations for clearinghouse 

operations, in general, are 'more likely to be maintained or reduced than to be 

/ncreased. Also, given that asr much as one-third of clearinghouse operating costs 

arft'spent on publicatipns programs, it is likely, that in many cases this function will 

'absorb some. of the budget modifications. It clearinghouses' current information. 

analysis and publication roles are to be recast, several options exist: 

' » ~ >• 

• . the current level of clearingho^e publication efforts can bQ 
reduced^ 

• costs for publication 'production and/or dissemination can b^^ 
recovered^ or . \ 

^ • the use of cooperative ^arrangements with private publishing 
• organizations Qan be increased. 

"^ Reductions in Clearinghouse Publication Programs , 

- Some clearinghouses have reduced their publication and dissemination efforts in 
response to previous budget decreases. 'Further reductions have beerf imposed more 
recently by 0MB Bulletin No. 81-16, which calls for a moratorium and 
comprehensive review of current publications activities* The Bulletin requires that 
publicatidns not essential in accomplishing agency missions be eliminated and that 
the number of copies made for essential publications be reduced. 

Specific ways in which clearinghouse publications programs have beep reduced 
in the past or could be curtailed in the future include; 

• eliminating particular- publications or types of publications 
considered less essential or those that have potential for being 
picked up by other organizations; 

I. 

• decreasing the number of periodical issues published each yeai^ 

* • 

• simplifying the 'format, graphics, and printing-^style -where 
possible; 

• reducing the number of copies printed^ and • ^ 

^ • limiting .the number of copies distributed to requester^ only, 
rather than automatically mailing copies to all names on a 
mailing list^ * ' ^ 



The advantages and disadvantages of each reduction st: Jtegy ' vary with specific 
circumstances and priorities within each clearinghouse. The extent to which 
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reductions can be Tnade without jeopardizing the effectiveness of clearinghouse 
publications and dissemination^efforts also varies depending on the clearinghouse's 
mission and previously imposed reductions. Decisions as to the most feasible and 
appropriate reductions must be made on a case-by-case basis. ^ 

The extent to which* Bulletin No. 81-16 will affect specific clearinghouse 
publications' is not** yet *knowQ. However, those p^articular publications that will 
continue to be produced by the clearinghouse but reproduced in limited numbers may^ 
be candidates for cooperative publishing arrangements with the private sector. 
Possible cooperative arrangements. are discussed later in this chapter. 

Cost Recovery 

• Recbvering costs of publications through user fees is called for., ih OMB's 
Bulletin No. 81-"16^ unless those publications are -specifically described in .and 
mandated by lavv. In addition; the 0MB draft circular on "Improved Management 
and Dissemination of Federal Information proposes that direct and indirect costs 
incurred in the printing, prqi^ssing, 'and retention of information be identified and 
that all costs associated with information dis^ernination be recoyered. 

Although very few clearinghoii^s currently recover all publication costs, most 
clearihghouses dp charge user fees ^for at least some of their publications, 
particularly for their' primary publications. Implementation of a more extensive 
cost recovery policy is likely to affect the criteria clearinghouses usie in determining 
whibh publications to produce and, in some cases, the' extent to which clearinghouses 
are able to promote their program objectives actively through their publications 
programs. Some clearinghouse directors are concerned that criteria for producing 
particular publications may Begin to reflect the publication's , marketability rather 
than its ability to meet user needs or to provide information for the public ^ood. If 
cost recovery were applied to publications that promote a government concern or 
priority (e.g., the hazards of high blood pressure or the societal costs of child abuse), 
the accomplishment of the mission of some clearinghouses might be seriously 
hampered. Further considerations in implementing a cost recovery policy are 
discussed in Chapter 3. * 

^ w 

cooperative Arrangements with the Private Sector ^ o ♦ 

Expanded use of cooperative arrangements with the private sector offers 
opportunities to accommodate user needs and accomplish dissemination objectives, 



while It the. same' time functioning at reduced funding levels end encouraging 
private sector activity. Increased cooperation between government and the private 
sector is recommended by the Joint Congressional Committee on Printing, the draft. 
OMB circula'r, and by Bernard Fry in a report for the National Commission on 
Libraries.and Information Sciences* 

As discussed earlier, some clearinghouses have >• established cooperative 
arrangements with commercial publishers, academia, or professional associations ' 
whereby publications produced by the clearinghouse are printed^jod- distributed by 
the private sector. These efforts are intended to facilitate wider dissemination^ 
clearinghouse publications at a reduced cost to the government and, in some cases, 
to foster good working relationships with the professional community. In addition,* 
.commercial publishers have initiated tbs ^publishing of non-copyrighted 
clearinghouse publicati5?(s that were viewed as having a broader market thgp would 
be reached through the normal government distribution channels. 

An example of the extent to which cooperative arrangements can be 

implemented for the benefit of both the government and the private sector is 

evidenced in the Market Linkage Project fo^ Special Education. The Marlcet Linkage 

Project way established in 1977 by what is now the Department, of Education's 

Office of Special Education "(OSE) to facilitate broader dissemination of bbbks, 

instructional materials, and educational media produced with OSE fands. These 

materials represent products that could not be developed economically by the 

commercial sector because the marketable addience is so thin. However, when the 

cost of research and development is subsidized by the* government,* commercial 

publishers are able to profitably reproduce and distribute the products. 
f 

In thfe Market Linkage Project, materials developed through OSE research and 
development activities are screened for their, marketability according to criteria of 
quality, 'Suitability to audience, market demand, and format. With"^t)proVal from 
the Joint Committee on Printing, items considered marketable are announced to as 
many as 500 publishers and producers as available for commercial publication. * 
Interested companies submit bids, or proposals presenting their production and 
marketing strategy and their planned corrvnitrrfent to the product. Bids arg assessed 
by an independent panel of reviewers, and a license to produce and distribute the 
product is negotiated with the most appropriate company. In addition, technical 
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assistance is offered to OSE research and development grantees in technique? that 
wiil maximi2:e the eventual marketability of those products. The Market Linkage 
Project is reportecjly post*eff active dissemination model, providing substantial 
• savings to the government, and^in some cases bringing* in additional royalties 
(i^arket Linkage Project for Special Educafi/rffV 1981:7). More thorough evaluations 
of the Market'Linkage Project arel:urrently being conducted. 

If commercial distribution of government-produced materials is shown to be 
viable and cost-effective, the Market Linkage Project or similar dissemination 
strategies may be transferable to clearinghouse publications programs. This model 
increases' reliance on the private sector, and provides an alternative to reducing 
current publications programs or incurring the additional burden of cost recovery. 

To explore the feasibility of implementing such a model, six representatives of 
commercial publishers and 'five representatives of Federally-funded clearinghocises 
were queried about their initial receptivity to the concept. Responses were mixed. 
Almost all of the clearinghouse representatives expressed interest in such a model 
and felt it was worth exploring. Some commercial publishers reported an interest in' 
getting more involved in publishing government products, whereas others expressed 
little interest. Some publishers feel that commercial dissemination and marketing 
strategies would provide more awareness, broader distribution, and better 
accessibility to clearinghouse publications than current disserhination activities. 
They fexplain that limitations on the distribution mechaniSms of 
government^supported disseminating organizations, such as information 
clearinghouses, NTIS, and GPO have resulted in less than adequate perform^ce in 
disseminating information to the public. Bureaucratic delays, funding restrictions, 
and lack of marketing efforts cause limited awareness of available materials. The> 
maintain that even users who are aware of a particular publication often cannot 
acquire 'a copy because cf the limited (lumber of printed copies or delays in 
processing and announcing its availability. They 'also maintain that if the private 
sector is interested in publishing government publications under available copyrtght 
opportunities, existing clearinghouse dissemination efforts tend >to^ weaken their 
potential market viability. Clearinghouse dissemination efforts typically provide 
copies to 'the core audience of interested recipients, thus limiting the total audience 
that is available to the private sector. 
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Commercial publishers agree that 'the major disincentives to involvement in 
cooperative- publishing arrangements are the compiicatrons; paperwork, apd time 
delays -caused by governnient^Blparance procedures. Private publishers interviewed 
consider completing gdvernment forms or submitting lengthy proposals, such as^he 
ones required tr^the Market Linkage Project, a major impediment. One publisher: 
specifically reported that if bidding forms or proposals were required, they should be 
no more than one page in length. Time delays involved in working with government 
publications are also described as problems for private puSlishers. Delays can be 
caused by the review process required to assess the? proposals submitted and by 
clearance ^procedures required by the Joint Committee oo Printing, if each 
publication licensed for privatQ publication requires clearance. Publishers who have 
previously published government publications report a six- to ninp-month turnaround- 
before the product can be marketed, in contrast to the 90-day turnaround tbey 
require for most other authors. Additionally, publishers express the need to receive 
jsole publishing rights if cooperative arrartqements are to succeed. T^ese "rights 
would have to preclude the non-co ^ghted publication of the same document by 
•GPO, the clearinghouse, or by other'^publishers. 

Of course not all clearinghouse publications are of a type that woulfJ interest 
the private sector. Few clearinghouse publications have "best seller" potential. 
Cooperative publishing arrangements would be applicable^only to those publications 
that fTieet marketability criteria. Such publications are more likely to include 
original literature, index and abstract publications; catalogs and directories, ;an,d 
some types of reviews and summaries. Bibliographies, factsheets, and brochures are 
less likely to be commercially marketable. Although- determining the4Tiarketability 
of a particular publication is somewhat, subjective, the private sector is more 
interesteci in puolications that: 

• are of compm^able— or better quality than commercially 
produced publications, with clearly stated objectives. and good 
internal organization; 

m have a practical content that is clearly presented and 
understandable b,y audiences ^t all levels; 

• have a scope that is neither too narrow nor too universal; . 

• have' an* acceptable format with rnore narrative than .tables 
and graphs 

• can be published without m^or editorial or format changes; 
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• provide master material that can be reproduced in present 

fornn; ^ 

• are sufficiently^niqu^ from other products availably and 

• have. been subjected to a pre-publication jeWew process among' 
^, (potential audiences and/or professional reviewers. 

More widespread use of these cooperative arrangements depends on the ability 
to match the dissemination needs of the clearinghouse with what the privdte sector 
can readily offer. .Those who have worked with alternative publishing arrangements 
suggest that, to be effective, three major factors must be considered. First, the 
organization or company interested in publishing ciearinghous^^ublications must be 
strong in marketing, . have high visibility in the field, and be equipped and 
experienced in handling the type ^and size of publication to be published. An 
association's membership or a commercial company's' marketing arena must be the*^ 
same or encompass the same target audience as that of the cle0ringhouse. 

Second, if a publication is to be publishedj by a commercial publisher, the 
marketable audience must be broad enougt) to erjs'ure the publication's commercial 
viability. Often a potential market can be strengthened when specific mandates or 
funding directives are implemented that focus attention toward a particular problem*- 
area and provide funds to acquire supportive materials, in working with 'that 
problem. A clearinghouse itself can often generate a potential market by focusing 
the attention of its audience on a particular topic or issue. Once this attention is- 
sufficiently mature, the interest and demand for publications increases and the 
audience can be marketed more successfully by commarcial publishers. 

Third, it a profession&l association or commercial publisher plans to charge fop 
a publication, the price will need to take into account the target audience's ability 
and willingness to pay. For the arrangement to be a practical alternative, the* price' 
should be. comparable to what GPO or NTIS would charge oit^^ what the 
clearinghouse would charge in recovering printing and dissemination costs. In some 
ca^es, the private sector has been able to sell a publication for a much lower price 
than what GPO or NTIS would charge, whereas in other cases the price, has been 
comparable or much higher. One government research report that sold at NtiS for 
almost $5.^0 v^as also published by a commercial publisher and sold in paperback for 
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. $.95. On the other hand, a clearinghouse-produced directory that sold through QPO 
for $6.30 was estimated by a private publishing company to be a $45.00 value on the 
commercial market. 

If the Market Linkage Project model or similar, cooperativ^e publishing 
arranyements are' to be iQripIemented more widely, mechanisms need to be 
established to facilitate such implementation* The Market Linkage Project and 
clearinghouse personnel who have worked ,with joint publishing arrangements offered 
suggestions on how to facilitate these arrangements* Publishers who are initially 
reluctant can be encouraged by at least establishing reliable corfimunication 
channels, minimizing the paperwork and time delays, ensuring sole publishing rights, 
and developing more marketable publications. Centralizing the communication 
channels between the public and private sectors, by using the existing publications 
6ffices at the agency or department level rather than the clearinghouse level, may 

e 

be more efficient for the government and more convenient for the private sector. 
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EVALUATING THE PERFORMANCE OF CLEARINGHOUSES 
Introduction 

Federally, sponsored • clearinghouses operate in a public-service and 
educatjon-oriented environment. This gives them aome advantages over 
organizations in the information service industry. For example, they are not 
compelled to show a profit (a requirement that carf^sometimes lead to short-sighted 
planning and inadequate- attention to the quality of the service provided). On the 
cfther hand, government-sponsoredcl^aritighouses operate under some distinct . 
disadvantages, of which the most important may^^Be-ttje lack of good marketplace 
measures to evaluate the usefulness and attractiveness o?^"th^irpublibations and 
services. Thus, alternative measures that are appropriate for their missions- and 
special operating environments are needed to provide some level of accountability 
and 1a basis for ongoing planning. 

A number of accountability questions related to effectiveness, cost 
effectiveness, and benefits have been and continue to be raised regarding Federally 
sponsored clearinghouses and their activities. These questions are asked at two 
levels: the individual clearinghouse and a group of clearinghouses. Questions that 
are asked of a specific clearinghouse include the following: 

• Is this clearinghouse dping a good job? 

• ^ What good is this clearinghouse accomplishing? 

.. • Do members of the target audience know ^ about this 
clearinghouse and its publications and services? If not, why 
not? 

• If sofne members of the target audience are aware .of the 
clearinghouse but do not use it, why are they non-users? 

• How is the clearinghouse's information being used and what 
effect/does it have on the user or the situation that led to the. 
use of tne information? 
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• What are the users* perceptions of the quality and utility of 
this clearinghouse's publications and services? 

• How effective or efficient is this clearinghouse, compared to 
others? 

Sometimes the questions are posed, not concerning a single clearinghouse, but 
concerning a group of them. For example: 

• Are the clearinghouses sponsored by this agency doing a good 
job? c ■ ^ ' 

• What good is being done by this group of clearinghouses? 

• Are there areas 'in whicfi two or more clearinghouses are 
duplicating effort unnecessarily? 

• Are clearinghouses, in general, obsolete? Is there some better 
mechanism for providing the publications and services that 
they currently provideTL' 

These and similar questions have been posed by individuals in many different 
organizations, including the Congress, GAO, 0MB, and program offices of 
governnrient agencies. Clearinghouse directors often pose these questions 
themselves. 

Individuals in each of these organizations may have somewhat different reasons 
for asking these questions, however. For example. Congress or 0MB may wish to 
determine whether the clearinghouses are sufficiently worthwhile to warrant more 
or less Federal funds, or any at all. On the other hand, clearinghouse personnel may 
want this infofojation to support the planning process by helping thenrj.make the 
publications and services even. more responsive to users* needs. When the reason for 
asking thejquejtion varies, the type of information required to satisfactorily respond 
may also differ. The appropriate evaluation strategies and data — including the 
identification and definition of objectives, performance criteria, methodologies, and 
measureme^nts — vary, depending on the purpose for which the information will be 
used. One cannot describe the "best* evaluation data and procedures, except in the 

context of a particular set of evaluation questions, ob|ectives, and purposes. 

^' { 

Measuring the "value*^ of information and determining how information is being 
used have long been concerns in the field of information science and are now also 
topics of interest to some social scientists. However, neither field has developed a 
completely satisfactory technique for obtaining this type of information. When 
applying concerns about the value of information services and products to the 
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evaluation of clearinghouses, the measurement ^problem is intensified because 
clearinghouses perform many functions and produce ^veral different kinds of 
materials* in their effort to serve the public interest and support specific social 
programs. The state-of-the-art in evalua'ting information services and the variety 
of clearinghouse activities that could be measured necessitate that clearinghouse 
performance be measured at the micro level. At. this level, evaluators can focus on 
questions for which measurement techniques have been developed and they^can 
select specific clearinghouse activities of services- to examine. In discussion of 

methods and measures, we have assumed two. possible audiences and evaluation 

' " > ' 

purposes: (1) clearinghouse directors and Proiect Officers who are interested in 

assessing the performance of a specific, clearinghouse and identifying* areas for^s 

improvements; and (2) agency program managers and policymakers who are/ 

interested in evaluating and comparing groups of Clearinghouses. 

This chapter addresses the evaluation process generally and discusses how the 
. suggested measures can be used.. It* focuses primarily on information that can be 
us^d by clearingliouse personnel to improve the performance of the clearinghouse 
and by decision-makers who wish to c(^pare performance among a group of 
clearinghouses. Since information dissemination alone is not sufficient to effect ^ 
specific social pr attitudinal changes and outcomes (see Chapter 1), the discussion is 
limited to evaluation and measurement of neutraj^^formation-related activities 
(i.e., those that serve -^n awareness and reporting function). Jhe chapter iS; 
organized into the following sections: , - % / :^ 

^ . _i - ^ ' ' 

• . problems and issues m evaluation design; . 

• bjiirrent evaluation practice^ 

• use of readily accessible data to support evaluation; and 

• user surveys to obtain additional evaluation information. 

Problems and Issues in Evaluation Design ^ 

A number of factors must be considered in designing a clearinghouse evaluation 
that addresses the needs of the two audiences listed above.- Among the most 
important are: • • 

• the specification of evaluation objectives and performance 
criteria; • 

• the definition of clearinghouse "users'*; and 



• the appropriateness of comparisons between and among 
. clearinghouses* , 

^\ 

These factors are discussed on the fisllowing pages. 
Evaluatiqp Objectives and Performance Criteria 

Gathering evaluation data, particularly user-§atisfaction data, without a clear 
statement of objectives and performance criteria, ia often an inefficient use of time^^ 
and resources. Questions can be posed in a periodic survey or data can be collected 
on an ongoing basis, to assess levels of user satisfaction and/or to measure volumes 
of activity* However, without some a priori set. of performance standards or 
targets, results are highly susceptible to interpretation of what constitutes "goocf* 
performance. • - " 

Two examples illustrate this point. The ERIC Document Reproduction Service 

(EDRS) sold 17 million microfiche during fiscal year 1980. This is certainly a^large 

number, but does it represent good or poor performance for EDRS? If we know that 

the 1980 sales represent a growth of ovf^4DQ..p e_rcent compa red to 1970 salens (about 

4 million microfiche), we are in a better position to Interpret the 1980 data. We 

would have an even more concrete basis for interpreting these data* if we knew. 

whether this gi^owth rate matched, exceeded, or failed to meet the targets set for 
> 

1980 EDRS microfiche sales. A very different example is provided an 
independe^ survey of physicians in which it was learned that approxii*nateIy 60 
percent of the survey population was aware of a core document produced by the 
High Blood Pressure Education -Program. This percentage was considered to be 
"highly satisfactory." But what if it had been 50 percent or percent? To 
interpret any such figure properly, we need to know the expectations the program 
had for awareness of this publication within the chosen sample of the target 
audience. 

Performance targets are frequently specified in statements of work for 
contractor-operated clearinghouses, but these are primarily associated with 
process-related measurements (e.g., volumes of documents processed for inclusion in 
the system or timeliness in responding to requests). Similar types of targets are not 
typically established for specific outcomes (e.g., degree of penetration within^a 
target population or levels or user satisfaction)* To establish such targets, the 
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participants in** the evaluation process, including the clearinghouse or its sponsor, 
may need to invest in some preliminary research* and to track certajn key 
* per/ormance indicators over a period of time. . 

Definition of Clearinghouse Users » ^ 

The complexities involved irr defining and /lifferpntiating between tfirget 
audiences and users of clearinghouses were discussed^^Jf/^ Chapter 2. These 
complexities must be dealt with .not only in the-planning function but in evaluation, 
as well, because a basic design problem for certain types of ^evaluation studies 
involves identifying and sampling users '(or non-user^. Identification is a major, 
problem for many clearinghouse^'their mailing lists and records of requests* may or 
may ndt represent the population to be' studied. For example, the organisation that 
requests the information from a clearinghouse may -be a library or other 
intermediary organization that is obtaining information (e.g., a publication or a 
liter atqje search) on behalf of one or more other individuals the ac/tual users of 
the information. Or the orgaftzations and individuals that request and receivs 
information may repackage that information for further distribution to the ultimate 
or, at least, once-removed*users. -Vhpse ultimate users'may oi^may not be aware 
of the original' source of information — the clearinghouse. 

The specific definition of target audiences for each clearinghouse and the 
linkages they establish through Intermediaries must guide the process of defining 
appropriate strategies and methocJologies for identifying the population to be 
studied. What is appropriate for one clearinghouse may very well be inappropriate 
for another! ' ' ^ ^ 

Multi-Clearinghouse Evaluations and Comparisons 

Differences in target audiences and users and the information infrastructure in - 
the field in which a given clearinghouse operates are only two of several major' 
factors tt^at must be addressed in jiesigning evaluation studies that involve more 
tharr one clfearinghouse. Evaluation is complicated by the facts that (1) a shared set 
of taxonomies for users, products, and services has not yet been developed and (2) 
cost*- accounting procedures are not standardised. across clearinghouses. 

Our data collection activities indicate that policy decisions requiring thev 
development and use of common definitions would be extremely helpful in 



supporting evaluation studies of^ multiple clearinghouses. For example, two possible 
sets of generic classes of users are presented in Exhibit 6«1« Clearinghouses could 
elect to classify users by affiliation or institution, by occupation and position, or by 
both types of classes. Although some .generic framework is needed, it 'must 
accommodate the specific needs of any given clearinghouse. For exarjhple, each 
clearinghoifse needs the latitude to impose some greater specificity /-;^ithin the 
generic classes. In the health field, clearinghouses may well want to^ecily nurses, 
physicians, and other specific classes of health-service practitioners tb>provide an 
appropriate level of user-related detail for analysis. ' 

The taxonomies are important both for studies that involve analysis of 
internally maintained data and for studies that involve full-scale field research. For 
records that are maintained internally, obtaining the desired level of demographic 
data on requesters and users of clearinghouse services and publications is not easy. 
The establishment of prescribed classification systems represents a major first step 
that is needed to support clearinghouse evaluation. 

Current Clearinghouse Evaluation Practices 

' One can envision three types of clearinghouse evaluations relating to (l*) 
internal operations; (2) user satisfactior^ and (3) impact of clearinghouse services. 
Data to support any of these evaluation types can be gathered formally or 
informally, as shown in Exhibit 6.2^- In our sample of 27 human services .information 
clearinghouses^ the preponderant data collection methods were informal. Cost, the 
need for 0MB clearance, and the absence of a strong demand for more formal, or 
scientific methods contribute to the decision .to rely on informal techniques. 
Consequently most current measurement of clearinghouse performance relies on 
readily accessible data. 

Performance is measured^to keep clearinghouse administrators inforfhed about 
what' the organization is doing and how well it is operating. Although all 
clearinghouses collect some performance data, their efforts are frequently limited 
to whatever information the sponsoriqg agency requests. Some respondents indicate 
that, with the number of demjnds placed on clearinghouse personnel, it is difficult 
to find the time to^analyze data for perfdrmance measurement. 



EXHIBIT 6.1: GENERIC CLASSES OF CLEARINGHOUSE USERS ^ 

ILLUSTRATION - OF A TAXONOMY TO .SUPPORT 
MULTI-CLEARINGHOUSE EVALUATIONS . - 



User Classes, by Occupation or Position 

Administrator or manager 

Educator 

Legislator 

Librai;ian or information specialist /' ^ 
Health service practitioner ' ' . 

Researcher * ^ 

Social service practitioner 

Other professional services (e.g., lawyer, consultant) 
Trustee or board member 
Student - . " 

Not applicable; requested as a private citizen 



User Classes, fay Affiliation or Institution 



Sponsoring agency 

Government agency: ^Federal _State ^LQcal 
Legii.lative or judicial branch: ^Federal _SEate _Local 
University, college, or other post-secondary or adult education 

institution , 
Prerschool through high school: ^Administration 
^Instructional 

, Medical institution: JHospital/Clinic _R8search Center * 
4 Social service delivery institution or agency: ^Private 
^Publicly supported 
Business dr industry 
Association, society, or foundation * 

dther non-profit organization ^ . ^ 
Library or information center:, _Public_State 

^University/College ^School _Busines^Industry 

^Clearinghouse ^Commercial 
- Not applicable; requested as a private citizen 



EXHIBIT 6.2: EXA^4^LES OF DATA SOURCES FOR DIFFERENT TYPES OF 
.CLEARINGHOUSE EVALUATIONS 
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Type of Evaluation 


Informal Sources 


Formal Sources ' 




Internal Operations 


administrative data from 
a specific clearinghouse 


standardized 
admihistrative data 
• from one or several 
cleannghouses " 




N 

User Satisfaction 


unsolicited comments 
from' users 


large-scale user 
surveys or interviews 




Impact of . " . 
Information on 
User's Actions 


occasional unsolicited 
comment from a user 


large-scare user 
surveys or interviews 

o 



The information ^ most frequently recorded* relates to internal "processing 
capaSilitieSi services 'provided;''' choracteristiqs of users, and costs (Applied 
Management Sciences, Inc., and Cuadra Associates,' Jnc, 1981:9.2). These types of 
data are valuable ^|)oth to the clearinghouse director, who must allocate staff and 
nnonitor activities, and to the sponsoring agency, which wishes to ensure that funds 
are being spent in accor^lance with contract f. ovisions. i 

Measuring user satisfaction or the quality of publications oftpn requires that , 
the clearinghouse obtain feedback from users. Techniques for measuring outcome 
include both formal surveys and informal methods, such as letters and telephone 
calls from users. Insertion' of user response cards in clearinghouse publications and 
-searches, compilation of letters received from users, telejphone surveys of users, 
expert assessment of publications, and participant evaluation of workshops are 
commonly used techniques. Staff at all but three of the clearinghouses visited 
report .the useof^jat least cne of these technique^ the use of two or more evaluation • 
methods is reported by personnel at 16 of the 27 clearinghouses (Applied^ 
Management Sciences, Inc., and Cuadra Associates, Inc., 19869*4). 

Use of Readily Accessible Data to Support Evaluation 

There is considerable potential for evaluating clearinghouse performance using 
readily accessible data. AdministrativeuJata can be used*creatively to"* support more 
evaluative analyses of clearinghouse effectiveness and performance. Currently, use 
of administrative data for evaluation may be limited for two major reasons. First, 
some of the more innovative measures may .not have been shared among 
clearinghouse personnel. More importantly, there is no consistency from one 
clearinghouse, to another in the measures and taxonomies uaed. The lat.ter situatiqn 
is a more« substantiaP impediment to evaluation because it. precludes 
crossrclearinghou^e comparison^* For adnii^Hstrative data to support evaluation 
activity, resources will need to be devoted to developing consisten,t reporting 
formats and definitions. Suggested measures ^or the major clearinghouse activities 
are presented in Exhibit 6.3. 



EXHIBIT 6.3: ILLUSTRATION OF CLASSES AND TYPES OF READILY 
ACCESSIBLE DATA 

\ 




INFQRtviATIQN SELECTION AND PROCESSING 
Activity Levels 

1. Number of sources scanned or searched regularly for pertinent 
information and items 

2* Number cf exchange and review agreements established with other 
' organizations to support acquisitions 

3* Numtfer of contacts made (e.g., with previous or potential authoTs and 
^ with associat^ns) to solicit material for review 

4» Number of items reviewed • ^ 

5.. Number of items accepted for inclusion in the formal cpllection/database 

6. Num^ of items processed (cataloged, indexed and, if applicabloi 
abstracted) • 

Costs ' ^ 

I. Cost per item (e.g.^ for cataloging, indexing, abstracting; creation of 
computer-readable records) 

Effectiveness 

1. For. each update (i.e., items added toSthe database during a given time 
period), the currency of materials proce^^d: those published in the past 
month; the past three months p^st six months etc. . 

2. Correlation between proportion of requests, by topic are9,*'and proportion 
of items in the collection/database, by topic area 

3. Percentage of user requests that are fulfilled entfrely from the 
• collection/database; fulfilled only partially from the collection/database 

not fulfilled at all by the collection/database 

«» * 

' 4. Number of hours of usage and numbers and types of using organizations 
(for publicly available online databa^ses only) 
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EXHIBIT 6.3; ILLUSTRATION Of CLASSES AND\ TYPES OF READILY 
, ACCESSIBLE DATA (CONTINUED) \ ^ 



PUBLICATION PREPARATION AND Diy RIBUTION 

Activity Levels ' . ^ 

1. Total number of publication/ produced and publication type: 
bibliographic/referral; original analyses^of the literature; newsletters and 
other rmiti-purpose reporting, announcementi and analysis pieces 

' 2. Number of publications prepared by clearinghouse staff and number 
prepared by outside qojisultanfs 

3. Number of ^pre->ubli&ation reviewec§^ and number \ of favorable and 
unfavorable reviews 




4. . Numbers of sources, by type, used to select ^^^ublication^ topic 

5. Numbers and typea of channels used to alert potential users* to the 
'availability of a publication 

6. Number of copies for which requests are received by the clearinghouse, 
aijd comparison of this number with the number of ,copies in the initial 
press run 

7. Number of copies distributed by the clearinghouse • and by other 
organizations 

8. Average turnaround time for responding to requests for publications 
Costs ' ^ ^ 

1. Total cost per publication 

2. Cost per publication, by one or more of these categories: labor (interna! 
and external^ production; printing mailing ma/<eting ^ 

Effectiveness . • ' ^ 

it 

L Number of favorable and unfavorable unsolicited comments and letters 
received and the-areas of satisfaction and dissatisfaction, per publication 

.2. Number of favorable and unfavorable reviews in the published literature 
and ^reas of strengths and weaknesses reviewed 

3. Number of clearinghouse publications picked up for reprinting and sale or 
distribution through anotl)er source 



EXHIBIT 6.3: ILLUSTRATION OF CLASSES AND TYPES OF READILY 
ACCESSIBLE DATA (CONTINUED) 



USER SERVICES 



Activity Levels 

1. Total number of requests received during specified time period, and 
breakdown by mail, tefephone, and wall<*in 

2. Total number of requests for which responses were prepared versus the 
number of referrals 

3. Number of inquiries received, by type of user 

4. Numbofl^of responses by subject or topic area ^ 

T 

5» . Total, number of responses prepared, by type (e.g., information about the 
clearinghouse; general information about agency program; general 
question on information available on^a topic for which pre-assembled or 
readily identifiable packages of materials are sent; quick reference 
questions that take less than 30 minutes to fulfill; computerized literature 
searches, using the clearinghouse database and/or other organizations' 
databases; research problems requiring more than 30 minutes of staff 
Sresearch) 

Cost 

L Labor costs in all user services ^tivities 

2. Average time spent per inquiry 

3. Average time spent per inquiry, by type of inquiry ^ 

4. Range and average computer costs per search, on the clearingJ^Quse 
database and on other organizations' databases 

E ffectiveness 

K Average turnaround time between time sn inquiry is receiyed and the 
; response is mailed ttS^^the user , 

• 

2» ^ Number of unsolicited, favorable and unfavorable letters or calls received 
and areas of satisfaction and dissatisfaction regarding the response 
package to an inquiry 

3. . Number of users who have previously used the clearinghouse ("repeat 
users") 
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EXHIBIT 643: ILLUSTRATION OF CLASSES AND TYPES OF READILY 
ACCESSIBLE DATA -(CONTINUED) ; , " ^ 



^ 

MARKETING/PROMOTION 



Activity Levels . . ^ 

L Number of promotional activities undertaken and completed, including 
attendance of staff at meetings and conferences; exhibits staffed at 
iTieetings and conferences; speaking engagements by staff; training 
^ seminars conducted on use of clearinghouse advertisements; conferences 
and meetings for which special clearinghouse materials are preparec^ 
journal columns prepared about the clearinghouse; and mailings of 
promotional pieced about the clearinghouse or specific publications and 
services 

Costs 

K Labor hours involved in promotional activities 

2* Percentage of total budget used in promotional activities 

Effectiveness - * ■ 

• """"^^ • 

Estimates of target population (by class) aware of clearinghouse! through 
analyses of numbers of OBmes on mailing lists or number of organizations 
and other intermediaries h which mailings are regularly sent 

• 2* Percent and absolute growth in clearinghouse use defined in terms of 
numbers of inquiries for informatiorv/ numbers of different individuals 
requesting information and/or pubiicationsi ^numbers of publications 
ordered \ ' 

|v ^ 3« Numbers of organizations (e«g«i associationi foundations) that regularly 
/ use the clearinghouse Q$ a source of informationi refer users to the 
clearinghouse, or'"publicize" the clearinghouse 

4. Numbers of other information organizations (e.g., librariesi other 

clearinghouses) that regularly use the clearinghouse as a source of 

informationi refer their clients to the clearinghouse! 'or subscribe to 
clearinghouse publications 



er|c 
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The proposed measures provide a taxonomy for classifying puolications (see 
Item 1 in Publications Activity Levels) and user services (see Item 5 in User Services 
Activity Levels). A taxonomy for classifying users was presented earlier, in Exhibit 
6*h Another area for which a taxonomy is needed is cost information. 

The data generated from these measures can be used internally to track 
performance over time. If objectives are set in advance, :t is possible to assess 
whether planned^ performance levels are, being met. Where problems are found, 
pwrsonnel can perform additional studies to identify causes and to develop solutions. 
If the same data are collected for several reporting periods (e.g., quarters or years), 
trends can be foUovyed. 

X CI 

^Particularly informative indicators include the following: 

• the correlation between the proportiorr of requests by topic 
area and the proportion of items in the database or collection 

* fir each topic ares— an Indication of the "adequacy of the 
selection process and an indicator: of a possible need to more 
fully develop the collection on a particular topiq 

e nucnb^r of favorable and unfavorablie reviews of clearinghouse 
publications— an indicalor of the *^?isibility and qu-ility of 
clearinghouse publications; 

• number of clearinghouse publication^ picked up for 
reprinting— another indicator of their quality and visibility; 

e number of resoonses by subject area— an indication of "hot" 
topics that may warrant development of a publicatior? 

• number of repeat user5-*an indication of user satisfaction; and 

• percent of and absolute growth in clearinghouse use— an 
indication of clearinghouse visibility and user satisfaction. ' 

^ ' If all clearinghouses were asked to collect these data and to use a common 
taxonomy for generating the data, the results would also have utility for 
cross-clearinghouse comparisons* Such Information could be useful to DHHS and 
other agency decisiorvmakers who may have to make choices about ths funding of 

^rticular clearinghouses. It must be recognized, however, that there are some 
dangers in comparing numbers across fields. For instance, the number of sources 
scanned for potential pcquisitions may be a reflection not only of the level of effort 
for a specific clearinghouse, but perhaps also of the volume of literature in a 
particular field. Similarly, variation in the number of information requests received 
or the number of copies distributed for a document, may reflect, in part, the size of 
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the target population, as well as the extent to which the clearinghouse has 
penetrated the market. Interpretation of the data must be sensitive to such 
differences. 

Despite these limitations, the use of administrative data and unsolicited user 
comments has important advantages. These data are relatively inexpensive to 
obtain and can be collected regularly. In contrast, formal evaluation surveys tend to 
be more expensive and to yield results at only one point in^time. The advantage of 
formal surveys is that more complete information can be obtained about user 
characteristics, their preferences, and their 'assessments of the clearinghouse. >n 
addition, some questions such as reasons for not using a particular clearinghouse, 
can only be answered through^surveying the target population. The following section 
describes three possible survey evaluations. 

User Surveys to Obtain Additional Evaluation Information ^ 

User surveys can be designed to address many different topics of interest. 
Because of general interest in user satisfaction with clearinghouse publications and 
services, we have designed instruments that can be field tested in these two areas. 
A third instrument to .measure clearinghouse 'visibility and target audience 
penetration is also presented. These three types of evaluation and challenges 
associated with their design are discussed in the text; the survey instruments are 

presented in an Appendix to this report. 

a 

Publication Evaluations 

Clearinghouses often use other publication distribution channels (e.g., NTIS, 
GPO, the ERIC Document Reproduction Service, and professional associations) to 
supplement their own dir.ect distribution, thus creating some distance between 
clearinghouses and their readers. Despite this, most clearinghouses receive 
sufficient feedback through the mechanisms discussed in the previous section to 
sense the general response to a publication and to identify their "best-sellers." 
However, informal feedback lacks sufficient detail— particularly^ in the areas of 
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quality, utility, use, and norvuse^^to provide adequate support for the put)lications 
planning process. Measurement areas and \type8 of data That would' be helpful to 
clearinghouse personnel in planning publications have beenv^ translated into 
illustrative questionnaire items shown in tlie Appendix to this report* They include 
users' perceptions of the quality and utility of the publication, whether they used the 
publication in theif work, for what purposes it was used, how they learned of the 
publication, and whether they would Fecommend it to others. 

In designing an evaluation study to obtain data in these areas, a number of 
considerations rnqst be addressed. They are discussed briefly below. 

^ f* * 

Approaches to Publication Evaiuation * The design of the evaluation will depend 
on what questions it is intended to answer. Qne study objective may be to have a 
fairly large sample of target audience ^members review one^ or more publications, 
independent of specific personal needs for such information at that time* In this 
case, a publication may be'sent to a large sample of reviewers who are asked to 
comment on it and project how it might be useful to them. Another approach may 
be to learn from a sample of target audience members whether they have read one 
or more clearinghouse publications and, if so, whether those publications met their 
specific needs at the time the pCiblications were read. This approach provides 
information about the visibility of the publication and about recipients' responses to 
it. In a third approach, an evaluation form may be sent to known recipients either 
with the publication or a few months later. This method assures that respondents 
will know of the publication. 

Readers and NorvReaders: A Timing Problem . Timing is a vital aspect of any 
publication assessment. The person who requests a publication may not become a 
reader of that publication for some period of time, so an evaluation questionnaire 
may reach some recipients before they become readers. On the other hand, some 
recipients may be reached long after they have read the publication and they may 
not recall the specific publicatioi under review. 

In dealing with the recall problem, one approach that has been used successfully 
' in the'pasrirthe "recall prompter*' (Wangeq 1972). Color photographs or document 
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representations (e.g., copies of title pages, tables 'of contents, and' extracts or 
summaries) can be used to help a reader recall particular pvjblicatidns. It would still 
be necessary, however, to differentiate the responsef of those who are answering 
I'from memory" from those who have a'copy in nand at the time of^responding to the"^ 
survey questionSi — ^ i (^^^^ , 

r Dimensions ^oft Publication . Quality and Utility > Quality and utility can be 
difined in generic term^s (i^, as two ''generic variables) or in terms of the more 
specific variables that each comprises. Tne uniqueness and independence of these 
general dimensions has yet to be established. Utility to the user may be a function 
of quality, or perceived quality may be a function of utility to the user at the time 
the document is read. In an earlier survey effort (Wanger; 1972:VII-37), some results 
suggested that "the need fpr a document on a particular topic may supersede the 
need for a quality document." 

♦ • ' - • 

A Context for Interpreting Evaluation Results. As discussed earlier in this 

chapter, the interpretation of user-satisfaction survey results can pose a problem.- 

In addition to^he need for jgftcifyiFig a certain set of expectations against which the 

evaluation results can be compared, it is also necessary to develop analysis plans 

that help to focus the interpretation of results (e.g., in terms of performance/effort 

ratios). This concept involves comparing the value of >a publication from .the users* 

perspective to the level of effort involved in producing- it, to determine whether 

publications that require more effort .to produce receive a higher user rating. A 

similar approach* was preliminarily tested in a study for the National Center for^ 

Educational Communication (Wangei; 1972). In' that study a Visibility Index (to 

represent the degree of effort expended in promoting -a publication) and a Level of 

Effort Index (to represent the cost of the publicatiolb) were created. Either or both 

of these ^process measurements could be developed and compared to some overall 

indicator of the users' judgments of quality and utility. If 4tudy results show, for 

example, that high-cost or other .high-level efforts do not produce high quality or 

high utility assessments, the clearinghouse may need to change direction. 
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User Services Evaluations 

As indicated in 'Chapter 2| the user services area involves a wide range of ^ 
activities* The focus of this discussion is on clearinghouse reference services, one 
of the services most visible to users. A substantive evaluation question in this area 
deals with user satisfaction with the information provided by the qlearinghouse. 
This information, often referred to as the response package, is typically a 
bibliography and/'or set of documents that deal with a question that has been posed 
by a requester. Measuring uset^ satisfaction with reference services involves study 
of Qsers' perceptions of variables suc)i as: 

• appropriateness and relevance of the information to users' 
needs; 

• currency of the information received^ 

• attractiveness of the information package 

• clarity of response; 

• appropriateness of the quantity of invormation receivec^ and 

• ' usefulness of referral sources identified* 

In addition, it would be useful to know the purpose for which the inforqfiation was 
used and, where appropriate, the, users* perceptions of value relative to cost. Some 
of these and other measurement areas are illustrated in the sample set oft 
questionnaire items provided, in-the-Appendix. 

A broad user 'services survey is helpful in assessing tKe^adequacy of the user 
services staff, provided that careful judgment is exercised in interpreting the 
results. One clearinghouse director noted that it is desirable for user services to 
receive a positive rating, but that extremely high satisfaction ratings may mean the 
staff is spending more time than necessary responding to each request. The 
implication is that there is a trade-off between user satisfaction and efficient 

allocation of staff time that needs to be monitored regularly. 

■* 

For certain types of evaluation objectives, focused experimental studies may be 
more appropriate than surveys! For example, to evaluate more specifically the, 
quality of output* from a computerized literature search, requesters may be asked to 
judge the relevance of each citation in a bibliography and the relevance of actual 
documents referenced in that bibliography. This type of study can be used to assess 



6.18 



n 



the effectiveness of the indexing system or the quality of searches performed by 
clearinghouse personnel.^ A recent study of searched of several National Library of 
Medicine databases indicated that behaviors and attitudes of searchers had a 
.considerable effect on the outcome of the search. Failure to check on the adequacy 
of the search formulation during the conduct of the search, for instance, limited the 
utility of the results (Wangeri 1980:3). 

• An ^pprppch to evaluating user services is the use of an expert panel to review 
a sample of responses real or simulated user requests. The approach offers the 
potential advantage of providing feedback on the quality of clearinghouse services 
without the expense and time,, required to conduct' large-scale suryey research. 
However, the ability of experts to accurately predict users* responses has not been 
tested and is not likely to be high, since judgments' of user satisfaction tend to be 
quite subjective. It might be valuable to conduct' an experimental study comparing 
user and expert responses to determine the extent to which experts can predict 
users* assessments. A 1968 study by Cuadra suggests thit great care would be 
heecfed to select and, ^define the experimental conditions. The study found that 
judgments of relevance were influenced by numerous factors, including the skills and 
attitudes of thef'judges, the instructions given, the setting in which the judgments 
took. place,. and the rating scale used (Cuadra, 1968;1*12). 

"Visibility" Evaluations 

* Some clearingho\jses are constrained through budgets or policies that limit their 
investments ia promotional activities and prohibit active image development (e.g., 
through 'promotional mailings or use of distinctive logos on publications). But even 
these- clearinghouses should make some effort to periodically assess the degr'ee of 
awareness within their target audience of the clearinghouse and of its information 
resources and ser\/u:es. 

This type of Evaluation, more than any other area already discussed, is a prime 
candidate for an ihdependent, full-scale evaluation study. Ongoing mechanisms are 
more readily developed (or are already in place) for micro evaluations of specific 
user services and .publications, but assessment of clearinghouse visibility and target 
audience penetration is exceedingly difficult without some external data-gathering 
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efforjt. One major objective for such a study would be £b identify and characterize 

non-uWs iue.f individuals within the target population who are not aware 'of the 

• » * 

clearinghpiise or, if aware of it> have chosen not to use its services or obtain copies 
of its publications). ; ^ ^ 

A majoir methodologicarconcern in developing sjiich a study is tti.e identification 
and sampling of an appropriate survey population. This process is particularly 
important for clearinghouses that operate under the constraints mentioned above 
because their limited promotional activities^make the use of their publications and 
services highly dependent on the Users* seeking them out or on the dissemination^ 
activities of inj^ermediaries. Clearinghouses that actively seek* tojdevelop awareness 
and an ima^e with intermediaries and ultimate users' -* may find it appropriate 
and highly desirable'^to develop a broad based sample population. They will then be 
able to assess' success within the more immediate target audiences and begin 
measuring "ripple" effect among users typically served by those intermediaries. 

A set of questionnaire items that might be used in conducting such , a study is 
presented in the Appendix. For any given clearinghouse, the questions cart be 
tailored to its specific activities and to develop a. set of respondent background 
questions that capture demographic characteristics and information needs, as well 
as user characteristics that are most appropriate for its operations. In cases where 
intermediaries are likely to comprise a major portion of the survey population, it 
will also be useful to add questions (or, alternatively, to use a case-stucly approach)* 
to quantitatively and qualitatively assess how intermediaries use clearinghouse 
services and publications in serving their own clients. 
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CONCLUSIONS AND RECOMMENDATIONS 

• Introduction * " 

The conclusions and recommendations piesented in this chapter derive from the 
research conducted in Phase III and from the study team's basic understanding of the 
role end operations of human serviced information clearinghouses. They represent 
. the opinions of the study team and are presented to the Department of Health and 
Human Services for its consideration* 

It has been noted in the past that there is an absence of high level agency 
involvement in the management of information services and that coordination of 
information services is limited both within departments and across departments 
(GAO; 1979:3-8)\ We find that this condition continu&s to exist within DHHS ancj 
between DHHS and o^r departments. AlthougTi some efforts are now being made 
to develop bureiaiPlJfagency-wide policies (e.gj., the studies initiated by the Office 
of Human Development Services and the Public Health Service), no one has been 
assigned responsibility for , establishifig DHHS policies or considering appropH ate 
long-term actions. Vflthout a group or individual specifically designated to consider 
the options presented in this chapter, it is unlikely that coordinated actions will be 
taken. One possible strategy is' to continue the internal panel that has, advised this 
study and to assign it the task of reviewing these suggestions. Alternatively, an 
individual or a r.ew committee could be designated to perform this work. 

Furthermore, several of the options presented in this chapter require change? in 
legislation that would affect information services throughout the government. • A 
higher-level body with responsibility for government-wide policymaking will need to 
consider these options and their implications. Since the Office of Management and 
•Budget and the General Accounting Office are currently involved in studying and 
suggesting government-v/ide information policies, they may wish to consider these 
matters. * ' - 
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Although the body of this report is Arranged aroui^d specific policy topics, there 
^are broader policy considerations that should be addressed by the Department before 
it can set dirantions for such specific facets of operation as cost-recovery practices 
or publications programs. If policies can be established in the broad area of defining, 
the government's role in information services, then a framework will have been 
established v/*thin which to examine the more specific concerns. This chapter 
discusses the broader considerations first and provides recommendations relating to 
specific management and operational coricerns within that context. 

Establishing a Government Information Pqiicy 

petermining the appropriate role of goYernment in providirig information is a 

critical first" stepMn defining government infc?*matioh policy. Since its inception, 

the government has been active in collecting information and making it available to 

the public. As needs have changed and the capacity to produce and distribute 

information has become more sophisticated, government involvement has expanded 

without a concerted effort to define which types of activities are properly within 

the governmental sphere and which can or should be performed by the private 

sector. The result is Ihe situation described in Chapter 4, in Which -there are no 

« 

uniquely public or uniquely private markets, teuhnoiogies, or functions. 

As the amount of Federal funding to support information services becomes 
more limited, it is increasingly important to determine in which areas the 
government wishes to continue its involvement and to ^establish priorities for 
funding. Although most decisions will have to be made on a case-by-case basis, 
some guiding principles may be helpful. The following suggestions are offered for 
consideration. 

First, the objective of "information dissemination" must be'given more precise 
meaning within tlje context of defining the government's role in the provision of 
information s^pvices. The government, may become involved in, and current 
government-supportej^Information programs offer, the following types o^f services: 

t disseminating (i.e., collecting, organizing, announcing, and 
- sometimes distributing) data and reports produced by 
government agencies or with government support; 

• disseminating data and documents produced by commercial 
and non-profit organizations (e.g., journal articles, reports); and 



• analyzing and synthesizing the-^Uterature in various fields to 
meet the information neQds of different target audiences. 

The first decision point is to determine which of these services are appropriate 
government responsibilities* 

In identifying appropriate government information services, the services 
available from other agencies and the private sector should be considered. One can 
assert that it is appropriate for the government to continue its involvement in 
disseminating information projJuced with • government support. This is a role 
performed by such agencies^ as NTIS .and SSIE, as well as i.. any, clearinghouses, which 
^llect, organize, and announce the information to the public, and make it equally 
available to ah interested users. It might be possible for' a private ojganization to 
perform these services, and some private organizations dc, but there. is little debate 
about the legitimaqy *of these government services. -.Systematic synthesis and 
analysis of the literature in various fields, or facilitating the distillation process, 
may also be appropriate government services because few private secondary 
publishers assume thbse responsibilitias. Disseminating information originally 
produced by the private sector (e.g., journal articles or abstracts developed by 
private database prqc^ucers) is another matter, however. This is a service provided 
by many private organizations, and the private sector .tends to view government 
involvement as an encroachment on its area of jurisdiction. 

Once decisions are made'^concerning the pppropriate role for government in the 
provjsion of information services, it becomes necessary to establish priorit||s in 
terms of the subject or 'problem 'areas that the Department "of Health and Human 
Services considers most important for continued funding. This decision can be made 
in one of two ways: 

• by identifying and providing information services in those 
subject or problem areas that are of highest concern to DHHS 
agencies; or 

• by identifying those areas that have the greatest commercial 
viability and attempting to stimulate privlte involvecnent in 
these fields thereby conserving DHHS funds for priority areas 
that have limited commercial potential. ^ 

-> 

We believe the second alternative is preferable because it is consistent with stated 
government policy not to compete with private enterprise and yet it offers 
assurance that areas of priority concern to the DepartmeAt will be covered. 



Several techniques are available to foster greater reliance on the private 
sector* Before any 9overnment*supported information service is initiated, ' the 
sponsoring agency should be required to explore the feasibility of relying bn the 
private sector 'for service. In soma ciases minor 'modifications in existing services 
could _be made to accommodate Federal needs. Technique^.- that can be used to 
encourage private initiatives include ta)j incentives, matching grants, or partial 
subsicttes {ue.; seed money) tq encourage the private sector to eoter a particular 
field These rhethods might permit a gradual phase-out of government support for a 
specific infolmation service. To support ^iocument delivery services, the modex 
fallowed by ERIC and the Securities and Exchange Commission can be replicated. 
Theie agencies contract with private organizations to provide all document delivery 
services on a full*cost*recbvery basis. The contract ^^is essentially a jicense to 
operate; there, is no public subsidy of the service. This same approach may have 
applicability to other services. 

Another approach that should be explored is developing tertiary 
databases— databases derived from other databases— as a means for providing better 
access to- highly specialized data needed to support some- government programs. 
Because the large databases produced by ERIC, NTIS, NLM, and SSIE cover a 
number of disciplines and areas, the items in their collections that are relevant to a 
particular program area are perceived to bej'buried," and because these ^operations 
are^run outside the program area, they are perceived to be "remote." Tertiary 
databases respond to the need for more convertfent access. The private database 
developers with whom we spoke' indicated a willingness to allow development of 
tartiary databases if the copyright on citations selected from their databases could 
be assured. A revision of the copyright law is needed to provide this protection. 

In those areas where continued DHHS participation 'is deemed desirable, the 
next step' should be an analysis of the best structure for providing service. 
Information services can be provided by a single centralized source that 
disseminates information, from many agencies (e.g., the NTIS model)j/ Alternatively, 
a single source for information might be organised by Federal? department, by 
subject area (e.g., health or education), or by program area. Tine current system^ 
includes elements'of each approach, thereby raising questions about the efficiency 
of the overall design.. To determine whether consolidation would be more effective, 
one must consider both efficiency of operations and donvenience to users. 



The data we collected suggest that some economies of scale may be achieved 
through consolidation* They also suggest that there may be some disadvantages! 
particularly in terms of the difficulties encountered in combining ' exist ing, 
incompatible databases and in targeting sirvites to the needs of a variety of 
differ?)nt audiences* A definitive answer gbout^ feasibility and possible 
advantages of consolidation cannot be provided without further studies to: 

^ m develop consistent. and complete cost*accounting systems that^ 
permit objective measurement of the cost of- clearinghouses 
and other information services, both for those operated by the 
government and those operated by contractors; 

e assess the full cost of performing similar activities, such as ^ 
indexing or abstracting a document, through the large 
government^supported information systems (e*g*, NTIS and 
NLM) and at the clearinghouses, to determine which type of 
organization has lower per-unit costs; 

e determine the management^ and administrative costs of the 
current system and the {potential for savings through 
consolidation; " 

• examine the output of both types of organizations to assess 
differences in their ability to meet users* needs* 

Theaef studies would provide a basis for judging whether cost savings can be achieved 

ft 

'without substantially reducing responsiveness to user needs* 

In considering the relative merits of consolidating information services, it is our 
belief that greater effort should be directed toward reducing duplication of effort 
thgn toward eliminating overlapping content coverage. Overlapping content 
coverage serves primarily as a convenience to the user and does not* necessarily 
represent wasteful use of resources. However,* duplication of effort between the 
databases developed by the large government- supported information services and 
the database development activities of the clearinghouses appears to be less 
justifiable. This problem may be difficult to resolve in the case of existing 
clearinghouses, where resources already have been expended to design and develop a 
database. However, action can be taken to avoid the repetition of duplicative 
activities in the future. When new programs with an information c.ompcnent are 
established arid when exisjtjng information service organizations elect to automate 
their systems, they shoulflljbe required to explore the feasibility of working with an 
existing government-supported database. Through various cooperative - 



arrangements, it should be possible to take advantage of already existing system 
capabilities and to let each program area fodus on such activities as information 
analysis and targeted information dissemination** 

The decision points outlined above ar^ the broad information policy topics that 
must be addressed for the Department and for the entire Federal gov$>rnment. Only, 
after these broad decisions have been made can policy be established to address 
management and operational concerns pertaining to clearinghouses. 

Clearinghouse Management and Operations Polic ies 

As indicated in Chapter 2, designing and operating a clearinghouse involves 
making, a great many decisions about the objectives of the clearinghouset its 
organizational'structure, and the package of services it will offer. In this report, we 
have examined three areas in detail: planning considerations, the feasibility of 
recovering operating costs through user fees^ and the role of the publications 
pfogram. We have also addressed the topic of evaluating clearinghouse 
performance, to provide input into future planning at the clearinghouse and agency 
levels. Our conclusions and recommendations in these areas are offered on the 
following pages. 

Planning 

The primary planning need is* to define more carefully the mission (i.e., the 
fundamental purpose) of each clearinghouse, within the context of the program in 
which it operates^ In defining this mission and translating it into a set of specific 
objectives, program managers must understand what "information dissemination" is 
and what it can accomplish, so that they can determine whether a clearinghouse is, 
in fact, the appropriate response to the perceived need. If a clearinghouse is desired 
and appropriate, then objectives that are both measurable and realistic must be set 
as a basis for directing the clearinghouse and assessing its performance. 

When deciding where to ' place a clearinghouse within the. bureaucratic 
structure, greater consideration should be given to the implications of establishing a 
"programmatic^' clearinghouse embedded within a larger program activity as oppo^d 
to creating an "independent" clearinghouse that operates as a relatively autonomous 
entity. Each approach offers specific advantages that may be better suited to some 
situations and contexts than others. The choice of approach should be made 
deliberately, after, full examination of the implications of each alternative. 



There is a need * to reconcile fixed funding support for clearinghouses with 
expectations that clearinghouses will provide a 'certain level of service to their 
constituencies, particularly in handling reference and referral requests. There are 
no incentives in the present system for a clearinghouse to become highly visible, 
because it most likely could not absorb any increased demands. Therefore a conflict 
develops between the objective of disseminating information as widely as possible 
and the ability of a clearinghouse to finance broad distribution of its materials or 
expand its base of users. 

« 

The development of policies and guidelines regarding cost accounting practices 
for clearinghouses is another critical planning need. Thfere currently is no 
consistency in how costs are categorized, and cost data for clearinghouses operated 
by the government are particularly limited. Availability of complete information 
would support assessment of the relative cost of "in-house" versus "contractor" 
operated clearinghouses and permit selection of the most cost-effective approach. 
This information also is needed as a basis for pricing in any cost-recovery program 
that the government decides to implement. 

Cost Recovery 

Cost recovery is a complex issue because there ai*e'so many variables to 
consider in establishing a realistic policy. It is important to consider the type of 
information being provided and the purpose it is intended to serve, to help determine 
whether asking the target audience to pay for the materials will' negate this 
purpose. It seems that both the clearinghouses and their users are receptive to the 
idea of charges for document delivery and literature searches. Whether fees should 
be imposed for all clearingho'jse publications and whether their distribution should 
bVlimited only to tnose who are willing to pay, the fee are topics that kve still being 
debated. '^Answering these questions is difficult because there is no objective^^ 
information' about how users perceive or use these publications. It may be that free 
distribption through depository libraries . and to a very select list of key 
organizations is sufficient and that all othei^^users should pay for the publications 
that they receive^ • 
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If a decision to institute cost recovery in clearinghouses y/ere made, there still 
would be two policy questions to address: what categories of costs should be 
recovered, and what agency should collect and retain the fees? The first question 
has important implications for setting prices* If the government fosters greater ^ 
private involvement in commercially viable services (a direction we recommend), it 
is likely that the priority services that it elects to continue will rc 
partial government subsidy. Charging for the marginal costs incurred in providing 
services and disseminating clearinghouse publications may be more' realistic than 
charging the full costs. It must be recognized, however, that until laws are changed 
so that clearinghouses or their sponsoring agencies can retain the revenues 
generated from the sale of their products and services to support further activity, 
there will be little incentive to incur the expenses involved in implementing user 
fees* Consequently, we recommend that government*wide acjtion be taken to 

'correct legislative impediments to cost recovery. 

f 

i 

It is necessary to institute two additional activities to support a cost*recovery 
program. The first, as suggested earlier, is the development of a cost^accounting 
system that is capable of producing data on all the costs associated with specific 
clearinghouse functions. The second is a commitment to building the marketing 
capacities of the clearinghouses so that they can generate the usage levels needed 
to support a viable cost^recovery program. This step will require a reorientation of 
priorities for program sponsors and clearinghouse personnel who, traditionally, have 
been guided by a public service orientation. ^ 

Clearinghouse Publications Programs 

Most clearinghouse publications receive only a limited distributiorv-typically no 
^ more than a few thousand copies-* because of the restricted marketing and 
distribution budgets of clearinghouses. If these publications are truly serving a 
need, then their limited circulation and their lack of availability through the 
communication channels most target audience members are familiar with (e.g., 
journals and commercial publications) are substantial impediments to reaching 
potential recipients. We suggest that a* comprehensive planning process be 
instituted for claaringhouse publications programs so that the need for, and quality 
of, the documents is assured and^the probabilitity of broad distribution is enhanced. 

Although all clearinghouses plan aspects of their publications programs, few 
utilize the most systematic and cost-effective procedures. Such a planning process 
Q would include the following considerations. 
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!• Conduct a formal needs 'assessment to identify topics that are 
considered government priorities or are of considerable 
interest to the target* audience but are not adequately 
addressed in available materials, 

1. ^Determine wRether other channels, such as commercial 
publisher's, academic presses, or professional journals can 
address the topic. 

3, * If any of these channels is appropriate, .encourage and 

facilitate publication development through tKat channel by 
providing bibliographic and referral services. Consider 
production by the clearinghouse only when no other source is 
t feasible, \ 

4, ^ When the clearinghouSte elects to produce a publication, select 
* authors through a competitive process. The author should be 

an expert on the topic; iiVhouse staff m^ay pot be appropriate, 

5, ^ Determine the most appropriate format (e,g,, journal article, 

monograph, or pamphlet) for presenting the information, 

, ' 6,' Conduct an extensive review of the* draft publication^ experts ^ 
and target audience members should be asked to comment on 
the docuiVient to ensure that it ,m6ets high -standards for 
accuracy, clarity, and appropriateness to the intended 
audience. The National Institutes of Health, Health Message • 
Testing Service has developed review procedures that may be 
adapted for this purpose, 

7, Select, the distribution channel that will ensure the widest 
circulation at the least cost to the clearinghouse. Consider 
refereed journals, cooperation with ^a professional association, 
and commercial publishers as alternatives to printing^ and 
distribution by GPO or the clearinghouse. 

Working with commercial publishers requires an understanding of their needs 
■ and timelines, streamlined procedures, and a reliable, centralized point of contact 
between the government, and the publishers. The Market Linkage Project operated 
by the Office- of Special Education offers a mbdel DHHS may wish to study fpr its 
/ applicability to clearinghouse publications. 

Evaluating Clearinghouse Performance 

The information currently reported by clearinghouses is insufficient for fully 
assessing their performance. One reason is the absence of comparable data across 
clearinghouses. In addition, little attention is devoted to^ analyzing ^he statistics 
that are compiled, the ability to measure performance can be substantially 
improved by modifying current data collation activities. The Department of 
Health and Human Services should invest in: 
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• deyeloping key evaluation qQestions— he*, defining what needs 
to be learned about the performance of a clearinghouse preset 
of clearinghouses, how the information> is to Be Osed, ancl by 
whorr? 

• developing common taxonomies for users, publications, and 
services, particularly if cross-clearinghouse comparisons are 
'to be made; ^ v 

• developing a cost-accounting system that will permit 
determination of the full costs associated with each function 
and will provide comparable cost data across clearinghouse^ 

• directing all clearinghouses to report the same performance 
data; and - ^ 

• compiling data already availalsle in /the clearinghouse(s} and 
analyzing these data, to identify baseline data for establishing 
performance targets and for developing hypotheses to be 
tested in the field* 

These actiqns will provide an information base capable of supporting analysis across 
clearinghouses and over time* Evaluators still will need to take into account the 
unique features 'and characteristics of the clearinghouses being compared, however* 

We believe il^is important that attention be given to the ongoing evaluation 
processes and that adequate budgets be. allocated to the planning and evaluation 
functions* To be sure, some questions, particularly those pertaining to users' 
perc^tions, can be answefed only through survey research, and a complete 
evaluation is likely to require information of this type, as well* But evaluation 
surveys— unless designed spe^cifically as longitudinal studies— capture datia at only 
one point in time. Ongoing evaluatiqn, based on administrative data and unsolicited 
feedb^ack^is critical to the year-by-year planning effort, to helfi^ clearinghouses 
remain responsive to changing environments and user needs* 
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PUBLICATIONS SURVEY 

The publication "that we would appreciate your reviewing- is identified and 
described below. 



Insert brief publication 
description here 



1. Because publications can b6 used in a number of different ways, it 
will help us first to know how you have used this publication. 
Please check the one response below that matches your use. 



r~| Have read the publication with a particular -purpose in mind 
(pleasOescribe purpose*^) ' ^ 



(60 to question 2) ^ 

* 

( I Have skimmed the publi'catl'on with no specific use in mind at 
this timer (Go to Question 2) 

I 1 Have not yet read or skimmeti the publication. (If you are. 
unable to skim or read the publication within- the next week, - 
please go on to Question 3 of this questionnaire.) 

2. The following chart identifies a numbev of different areas of the 
potential quality and usefulness of publ ications. For each of the 
.areas" identified in the left-hand column, please check the rating 
that best represents your evaluation of the publication. The 5-point 
rating scale represents the following adjectives: 



4 = Not very 

5 = Not at all 



1 = Extremely 

2 » Very 

3 = Acceptably 

You will need to read these rating-scale Choices in the context of 
each particular area. For example, the choice points for the first 
area would be read as follows: 1 = Extremely authoritative; 2 - Very 
authoritative, etc. Please provide your considered judgment for each 
area, using the central choice point (3) only when it best reflects 
your opinion, and using the unnumbered responses only if necessary. ' 



143 



QUALITV 
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If you have not read or skimmed the publication, please indicate why. 
(Check the one response 'that most accurately describes your reason J 

( I The publication has just arrived. 

n I- have .not yet- had time' to review the publication but plan to review 
it in"" the future. 

< am 

■J 

I [ The publication arrived too late to be useful for my needs* 

I I The publication does not seem sufficiently oertinent^to my needs or 
interests to warrant review, 

I I Other (please specify) \ 

(Go on to Question 4) " 

Are you aware of publicjitions on' this topic produced by sources other than 
this clearinghouse? 

I \ vis (Go on to Question 5) 
r I No (Go on to Question 6) 

If yes, would you rate this publication in comparison to oublications 
produced by other sources as: 

j I A very useful publication 

I i Not unusually useful, but worth having available 
I I Not sufficiently useful to justify its publication 
\ j No opinion 

(Go on to Question 6) ^, 



6. If you had not obtained thH publication from the clearinghouse, what 
would you have done? (Check as many *esponses as appropriate.) 

[ t 6on3 to^the library to obtain materials 
I ) Asked colleagues for information 

IJ Gone to another clearinghouse in the field for assistance 

Relied on other resources already in my possession 

No other resource would have been available to me 

i \ The nee^ for this publication was not very great and I would not have 
made any effort to obtain information from another source 

. cm Other {please ,specify)_^ . 



7. As a result of reading the oul^li cation, did you use the information or the 
publication- in any of th^ following ways? (Check as manv responses as 
appropriated 

( I Used the. facts or recommendations to make a decision 
Ln"^ Applied the Information in my own work 
rn Used the information to give advice to. other people 
[ ] Examined other documents 

I I Consulted with author(s) or other persons identified in the 
publication 

I I Passed the publication on to colleaguefs) to read 

r"T Other fplease specify) ^ 



The following questions are intended to ^provide some background information on 
yourself and on your awareness and use of the'clear'^nr^nouse that oroduced the 
publication that you have just reviewed. 



8. Check the one choice below that best and most specifically represents the 
type of organization, in -which you work. In a few cases, you are also 
asked to checR another more specific response. (Please read all choices 
before making your selection. If none of them applies, olease check . 
"OTHER" and describe your organization. If receipt of the clearinghouse 
publication was not associated with anj^ professional .responsibilities, 
please check "Received as a private citizen".) 

□ Government agency (and, also check one): □ Federal □ State □ Local 

□ University, college, or other post-secondary institution 

Q Pre-School through high school (including all administrative and 
instructional units) 

\ □ Medical institution (e.g., hospital, clinic, or research center) 

Q Public or private social service delivery agency 

□ Business or industrial firm 

Q For profit research* organization or consulting firm 

□ Not-for-profit research organization, association, or foundation 

' □ Library or information cfifnter (and also check one): □ pubTic Q state 

□ university/col Ifege Q school □ business/industry Q clearinghouse 

□ commercial information broker Q other: 



□ Received as "a private citizen, for non-professional use 

t 

□ other (olease specify): 



9. Check the one choice below that best ^scribes the capacity in which you 
requested "tlTTs information. (Please *lect the most^specific description 
and the one that* best describes your primary responsibilities relating to 
the use of this information.) 

□ Legislator* 

Q Administrator or manager in an organization (e.g., -a Federal, state or 
local governmental agency, non-profit agency, or educational 
institution) 

[J Educator (e.g., teacher, instructor, professor) 

□.Health services practitioner (e.g., nurse, doctor, clinical 
psychologist) I 
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Q Social/:services practitioner (e,g,, social worker, counselor, case 
worker) ' , - 

}[2 Researcher ' ^ * • 

Q Librarian or information specialist 

□ student.. 

^ O Requested/received as a private citizen 

Q Other (please specify): 

10* Indicate below the frequency with'which you have initiated contact with- 
the clearinghouse that produced this publication, 

Q I have ;iot used this clearinghouse before; I obtained this document 
from another source, 

□ I had my first contact with this clearinghouse when I- requested this 
publication, 

p] I have previously requested information from this clearinghouse, 
(If this response is checked, please also check one of the choices 
•below,) 

Q One previous request 

ri ^ to 5 previous rcKjuests 

^ Q Over 5 previous requests 

L) I do not recall the number of previous contacts with this 
clearinghouse 

11, How did you 'first hear about the availability of this publication? > 

^> □ I did not hear Ibqut it^di recti. y; I obtained a cooy fronr (please 
check below): 

Q our library ^ Q another clearinqhouse 

.D ^ professional association □ ^^^^^ (please specify): , 

Q 4 colleague ^ . 



n I learned about it in the following way fplease check below): 

□ from newsletter or other announcement sent by the clearinghouse 



Q] from newsTetteroT>otner announcement sent by some other 
organization---'^""^ 

n from -a column in a journal that I read 

Q at a conference or meeting 

Q as a result of an inquiry I 'made at the clearinqhouse 

□ it was sent to me without my soecifically requesting a codv 

Q other (please soecifyh . 'Z 



Thank you for .your cooperation in this study. 
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USER SERVICES SURVEY , 

1. Please indicate the purooses of .your request for information from the 
clearinghouse, (Check as^'many as are aoplicablej 

^. . 

• O To update my knowledge in an area that I am studying or researching 

PJ To obtain an overview of a new area with which I have little or'no 
familiarity 

□ To obtain some soecific background information that wi>l hevlo me to 
solve a personal problem > ' 

\Q To obtain some specific background information that will helo me to 
X^solve a professional problAi that I 'am researching 'or studying 

Q to. obtain a specific fact or answer ^ . . , 

Q To identify individuals who. are working iri this area 

• CD To** identify relevant research in this area • 

Qj To identify programs, projects, or other types of resources in this 
area ' . ' . • 

'Pl Other (please specify): . [ 

2. Did the information you receive irieet the ob^jectives you checked above? 

□ Yes, -fully □ Yes, but only' in oart □ No Q Not sufe 

3. After obtaining the clearinghouse response to your question, did you use 
the information in any of the following ways? (Check as many as are ; 
applicable.^ 

Q I read one or more of the documents that were 'referenced /or provided!. 

r~j I. used the Information to give advice to others. 
'"^ ! ■ • • 

Q I consulted with at least one other individual who was identified 

through the clearinghouse information I received. . 

Q I passed* the Infofrmation from the cTearinghouse *on to one or more of 
my friends or colleagues. 

; Q Other (olease specify): ' [ 



Plpase comment on the overall usefulness of the- information that you 
received from the clearinghouse. 



The following chart identifies a number of different areas of cmality and 
usefulness. For each of the areas, identified in the left-hand column, 
please check the rating that best represents your evaluation. The 5-point 
rating scale Represents the following adjectives: 
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1 • fxtrttisly 

2 •'very 



3 • Acceptably S - Not at tP 

4 • Hot very 



You «ni need to read these rattnj-sctle cbokes In the context of each particular area. For example, the choice points for the first area would be read as, 
follows: 1 • Extre«ely courteous ami helpful; 2 * Very courteous and helpful, etc. Please provide your considered Judgnent for each area, using the 
centj^l choice point (3) only Uien it best reflects your opinion, and using the last three response choices only If necessary. 



ASPCCTS Of SERVia 



Extreaiely 



2 

Very 



Accept^ly 



4 

Mot Very 



RATINGS 



Not at an 



No opinion 



Do not 
Recall 



Not 
Applicable^ 



'(For telephone Inquirtes and 
site visits ooly) 
Courtesy and helpfulness 



Acceptability of lapsed 
tl«e before an answer 
was received 



Overall appropriateness 
relevance) 'of response 
0 the Inquiry 



Accuracy of the Inf onaatlon 
provided 



Up-to*dateness (currency) 



(If appropriate) Attrac- 
tiveness In fomattlng of 
package that was fecelved 



(If appropriate) Appro- 
priateness of the MMbec of , 
references that were Identified 
to ay needs 



(If appropriate) Useful.')ess of 
references In Identifying and lo- 
cating full-text source docuaents, 
AV aaterlals. Individuals, etc. 



(If appropriate) Quality 
of service received fro« 
organl2atfon or Individual to 
Mho« the clearinghouse referred 
•e for an answer or additional 
(nfonaatton 
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5. Would you use thd clearinghouse ag^in to obtain information in the area 
that it covers? ! 

□ Definitely O'Likely □ Not Sure □ Not Likely □ Definitely Not 
^ooments: ^ 

•r 

t 

7. Check the one choice below that best and most specifically represents the 
type of organization in which you work. In a few cases, you are also 
►asked tp^check another more specific response. (Please read all choices 
before' making your selection. If none of them applies, please check 
'■"OTHER" and describe your organization. Your information request was not 
associated with any professional responsibilities, please check "Requested 
- as a private citizen".) 

Q Government agency (and, also check one): □ Federal □ State Q Local 
Q University, college, or other post -secondary institution, 

□ Pre-School through high school (including all administrative and 
instructional units) 

□ Medical institution (e.g., hospital, clinic, or research center) 
Pub>ic or private social service delivery agency 

[j Bu'siness or industrial firm 

[j For profit research organization or consulting firm- 

[j Not-for-profit research organization, association, or foundation 

□ Library or information center '{and also check one): Q public □ state 

□ university/college □ school □ business/industry Q clearinghouse 
[1 commercial information broker Q other: 



[J Received as a private citizen, for non-professional use 
[j Other (please specify): 



si 
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8. Check the one choice below that best describes the capacity in which you 
requested this information. (Please select the most specific description 
and'the one that best describes your primary responsibilities relating to 
the use of this information.! « 



□ Administrator or manager in an organization (e.g., a Federal, state or 
local governmental agency, non-prpfit agency, or educational 
institution) 

[]]] Educator (e.g.-, teacher, instructor, professor) 

Q Health- services practitioner (e.g., nurse, doctor, clinical 
psychologist) 

Q Social services practitioner (e.g., social worker, counsielor, case 
worker) 

ri Researcher 

Q Librarian or information specialist 
L] Student 

Q Requested/received as a private citizen 

Q Other (please specify): ^ - 



9. Indicate below the frequency with whi.ch you have initiated contact with 
the clearinghouse that provided this information. 

P I had my first contact with thi^ clearinghouse when I requested this 
• information. 

□ I have oreviously requested information from this clearinghouse. 
(If this response is checked, please also check one of the choices 
below. ) 

n One previous request , 



Q Legislator 




requests 



Q Over 5 previous requests 

Q I do not recall the number of previous contacts with this 
clearinghouse. * 
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10* How did you first hear ?(bout the ex:J^^tenc&i*6f this clearinghouse? 

/ -/^""'^ 'V^^ 

/ \ .A 

Q f rom newsletter or/other announcement sent by the clearinghouse 

□ trom newsletter Or other announcement sent hv some other 
organization 

Q from a column in a jouV^nal that I read 

Q] at a conference or meeting 

Q as a result of an inquiry I made at another information service or 
library 



Q other (please specify): 



Thank you for your cooperation in this study. 
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' , VISIBILITY SURVEY 



Description of Clearinghouse 

(include logo* or other 
distinctive text used regulary 
clearinghouse announcements/publications^ 



1. Please indicate below your familiarity with the clearinghouse that is 
described above/ (Check only one response.^^^ - . ' ^ 

□ RESPONSE A ; I am familiar with this clearinghouse and its 

. publications arid services, fif checked, please proceed to OuestiQn 2 
and continue.) : 

□ RESPONSE B ; I have heard^ that such a clearinghouse exists but have no 
personal knowledge about *its publications and. services. (Please skip 
to Question 6 and continue.) 

□ RESPONSE C ; I did not know that such a clearinghouse existed prior io 
reading the description provided above. (Please skTp to Question 6 
and continue.) 



questions; 2, 3, 4 and 5 SHOULD BE ANSWERED ONLY BY THOSE WHO CHECKED 
"RESPONSE. A" ABOVE. 

2. Check each of the response choices below that applies either to your 
knowledge of or use of the-clearinghouse. (Check as many as apply.) 

r] I read newsletters and other announcements that are sent to me by the 
clearinghouse 

pi I have contacted this clearinghouse for the following reasons 
(check as many as apply): 



to order one or more publications 

Q to obtain the answer to a specific question 

[j to ask that a search of the literature be conducted on a tooic of 

my choice 
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n to sign up for a conference of workshop that it was sponsoring 
□ other (please specify): ' 



□ I read publications, that are sent automatically by the clearinghouse 

□ I have read one or more of its publications, although I 'generally 
obtain them from some other source 

□ I sometimes hear about the activities of this clearinghouse from my 
, colleagues* & ' 

n I have sent the clearinghouse at least one document (e.Q,, paper, 
article, or report) that I prepared, ^for inclusion in its collection 
or database 

Q I have'received requests from the clearinghouse for me to submit 

documents fe.g,, papers, articles, or reports) that I have written, ^ 
^ut I have not yet submitted any 

Q Other (please specify): ! _ 

If you have contacted the clearinghouse directly (i.e., you checked the 
second response choice in Question 2 above), olease indicate the frecwency 
of vour usage over the past two years. (A "contact" is defined to mean 
that vou have- asked the clearinghouse— in writing, in person, or in a 
telephone call— for information or for a publication, you have signed up 
for a conference br workshop, or you have otherwise initiated some 
request.) ; 



Q Have had only one contact in the past 2 years 

[J HaVe had between ?. and 5 contacts over the past ? yeai'S 

P Have had over 5 contacts over the past ? years 

[j Cannot estimate the number of contacts I have had 



*4, What is your overall assessmefftt of the quality of services and/or 
publications?' 

Q Generally very good 

I I Generally good " . - 

Q Generally fair 
n] Generally poor 

Q Can't really say; have formed no opinion 

[71 Can't really say; have had a wide mix of reactions— some good and 
others not 

5. If you are familiar with the , clearinghouse but have had no contacts with 
' its staff over the past several years (other than to receive its periodic 
"^"^Inai lings U please indicate the reasons why: 

□ Not applicable; have had contacts, as indicated above 

(n 'The area covered by the clearinghouse is only a peripheral interest to 
me ^ , 

(~) The procedures for contacting the clearinghouse are not clear 

r 

□ I havc .yet to hear about a publication or service that is* of specific 
interest to me or particularly relevant to my needs 

f 

□ I have not been entirely sure what kinds of requests the clearinghouse 
could help me with 

□ I simply have-not had the occasion to seek help from the clearinahouse 
Q I do not have any particular reasons 

Q Other (please specify): ' 

(Please skip now to Question 7) 



QUESTION 6 SHOULD BE ANSWERED ONLY BY THOSE WHO CHECKED "RESPONSE B"vOR . 
"RESPONSE C" IN QUESTION 1 ABOVE. 

6. Check below the response choice that represents your potential interest in 
the publications and services of this clearinghouse. (Check only one.t 



Q I am definitely interested in leattiing more about this clearinghouse 
' and plan to write for more information. 

n I would be interested in learnina more about this clearinghouse, but 
doubt that I wilKpursue -it further, because my information needs in 
this area artf being met through other sources 

Q I am unable to form any opinion on the fJOtential interest in the 
publications and services of this clearinghouse. (Check only one J 

Q I would be interested in learning more about this clearinghouse, but 
doubt that I will Pursue it further, because my information needs in ^ 
this area are being met through other sources. 

Q I am unable to form any opinion' on the potential usefulness of the 
clearinghouse ur>til I have learned more about it; however, I am not 
sure whether I will contact the clearinghouse for more information 

. Q I am not at all interested in learing more about this clearinghouse 
because (please check below): 



' O not relevant .to my area of interest 

. □ it is doubtful that I would ever use its publications or -services 
Other (please specify): 



7. Check the orie choice below that best and most specifically represents the 
type" of organization in which you work. In a few cases, you are also 
•asked to check another more specific response. (Please read all choices 
before making your selection. If none of them applies, olease check 
"OTHER" and describe your -organization. If your interest in the " 
clearinghouse is not associated with arv/ orofessional responsibilities, 
please check "As a orivate citizen".) 

□ Government agency (and, also check one): □ Federal Q State □ Local 
" Q University, college, or other post-secondary institution 

□ Pre-school through high school (including all administrative and 
instructional units) 



f 
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□ Medical institution Ce»^q., iiosp-ital, clinic, or research center^ 
Q Public or private social service delivery agency 

□ Business or industrial firm 

Qfor profit research organization or consulting firm 

Q Not-for-profit research organization, association, or -foundation 

□ Library or information; center (and also check one): □ public □ state 

□ unlvarsity/college Q school □ business/industry □ clearinghouse 
Q commercial information broker Q other: . , 

QAs a private citizen, for non-professional use 

Q^Other (please specify): 



8. Check the one choice below that best describes the capacity in which you 
^ might be interested in this information. (Please select the most soecific 
description and the one that best describes your primary resoonsibilities 
relating to the use of this'-inform'ation. V 

Q Legislator 

□ Administrator or manager in an organization (e.g., a Federal, state or 
local governmental agency, non-profit agency, or educational 
institution) 

□ Educator (e.g., teacher, instructor, professor) 

Q Health services practitioner (e.g., nurse, do.ctor, clinical 
psychologist) 

□ Social services practitioner (e.g., social worker,, counselor, case 
worker) 

Q Researcher 

□ Librarian or information specialist 

□ Student 

□ As a private citizen 

[[TOther (please specify): 



Thank you for your cooperation in this study. 
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